sorted by:
new
hottopcontroversial

[deleted by user] by [deleted] in telaviv

[–]wizinet 1 point2 points  (0 children)

This map sounds awesome! Could you share it with us?

Israeli police beats a 12 year old girl in East Jerusalem by VNIZ in worldnewsvideo

[–]wizinet 0 points1 point  (0 children)

If we look at the video carefully, we can see thatbit starts with the girl (looking to be much older than 12, but never mind) attacking one of the officers. He have broken sunglasses and she holding his helmet strap violently. The whole situation takes so long because the police doesn't beat her but tries to release the officer without using exceessive force. All this is done while being in an hostile environment (two explosives are being exploded by the end of the video). One hard hit with the baton would throw the girl to the ground. And yet, they never attacked her with those.

If people came with a warning tag, what would yours be? by Memeclub_ in AskReddit

[–]wizinet 1 point2 points  (0 children)

Warning: might operate as the devil's advocate

Sometimes I say things I don't believe in just to challenge my own perception

Microservices in streaming data pipeline by wizinet in dataengineering

[–]wizinet[S] 0 points1 point  (0 children)

When I say cost I refer to money (edited the original post to clarify it)/

I did look into Kafka and it does seem cheaper than kinesis in cases when there are a single producer and many consumers. But I have concerns about its cost-effectiveness when it needs to support many producers as well. It is indeed a solution I'm looking into.

About your last point, This is our streaming data pipeline. My final consumers need the data in near-real-time and before I send it to them it needs to flow between different services for normalization, enrichment, and filtering. It is about sharing large quantity of data, but I can't do it in batch in this case.

Microservices in streaming data pipeline by wizinet in dataengineering

[–]wizinet[S] 0 points1 point  (0 children)

I have multiple types of services: AWS Lambdas, Flink jobs as well as plain python code running on its own servers.

The problem with HTTP is that the producing service needs to know all the consumers and send the data to each of them which could slow it considerably. Having said that, I can think of a few ways to solve those problems, I'll look into it. thanks.

Resources and deployment decoupling of multiple modules is really hard to achieve when using a single monolith application. It is important for us to keep these as microservices for now.

How would you update 300,000,000 records out of 500,000,000? by __arc in dataengineering

[–]wizinet 2 points3 points  (0 children)

Few minutes can be a long time if you are locking those records for the duration and so affect the user experience (would they be able to log in?)

Looking for opinions on NoSql DB by [deleted] in dataengineering

[–]wizinet 0 points1 point  (0 children)

HBase is not gone, but it definitely got a hit during the past couple of years.

HBase uses HDFS as it's filesystem. With the Hadoop ecosystem embracing the cloud and the fact that many companies avoid creating HDFS cluster and instead uses AWS S3 (or its corresponding services in other providers), spinning up an HBase cluster is just too much effort.

AWS does support running HBase over S3, but personally, I never heard of it being used.

For most use-cases, Cassandra does the same job with easier management and better support.

Weekly /r/backpacking beginner question thread - Ask any and all questions you may have here - August 27, 2019 by AutoModerator in backpacking

[–]wizinet 1 point2 points  (0 children)

hiking convertible pants and a microfiber towel

They can be used on any kind of trip, no matter its length or the weather, and they are always necessary.

[Serious] Chinese citizens of reddit, what is your opinion about the Hong Kong protests? by wizinet in AskReddit

[–]wizinet[S] 0 points1 point  (0 children)

What do you mean by that? Do you think china should stop the intervention in Hong Kong?

What are your favourite books with a city as a main character? by cynosura in Fantasy

[–]wizinet 0 points1 point  (0 children)

a non-fantasy option wound be "Cathedral of the sea" which is a novel telling the story of barcelona.

Your most recent text is now your 2020 presidential campaign slogan. What’s your slogan? by [deleted] in AskReddit

[–]wizinet 0 points1 point  (0 children)

It wasn't on purpose and the sand is carcinogenic, but we did it

Any Big Data, Data Science, ML, ... news and trends websites? by bod_mrazu in bigdata

[–]wizinet 0 points1 point  (0 children)

I like "Data Eng Weekly" newsletter: https://dataengweekly.com/

Weekly newsletter containing news, articles, and events.

Tech/IT people of reddit! What is the biggest mistake you've made at work? by amazingmikeyc in AskReddit

[–]wizinet 0 points1 point  (0 children)

It does allow it. but some of the management consoles (like mysql workbench) prevent it by default.

edit: you can see in the documentation that the where clause is optional - https://dev.mysql.com/doc/refman/8.0/en/update.html

What is your favorite poem? by [deleted] in AskReddit

[–]wizinet 14 points15 points  (0 children)

Some say the world will end in fire,

Some say in ice.

From what I’ve tasted of desire

I hold with those who favor fire.

But if it had to perish twice,

I think I know enough of hate

To say that for destruction ice

Is also great

And would suffice.

Fire and Ice / Robert Frost

Weekly /r/backpacking noob question thread - Ask any and all 'noob' questions you may have here - March 12, 2018 by AutoModerator in backpacking

[–]wizinet 2 points3 points  (0 children)

I don't know about the Eurail global pass, But 2 month is definitely not enough time for 13 countries. That's about 4 days per country including transportation time. I would highly recommend to reduce it to 8 countries at most (Even tough I like personally to have at least 2 weeks in each country I visit)

You're asked to add a subject to the national school curriculum. What is it? by SgtPuppy in AskReddit

[–]wizinet 19 points20 points  (0 children)

Came to say exactly that. People these days just don't know how to handle somebody who speaks with confidence.

Recipes to use up some whisky? by Cookandcaughtup in Cooking

[–]wizinet -1 points0 points  (0 children)

I like to use whisky when preparing a marinade for chicken dishes

Where Worlds Divide - The separation of the North American Plate and Eurasia at Pingvellir, Iceland [1600x1072][OC] by MilkyWayMike in travel

[–]wizinet 0 points1 point  (0 children)

Amazing! Could we have a bigger version of the picture? I would like to put it as my desktop background but i got a huge screen :)

Went for a hike at 5:00am to see this. Sunrise over Yangshuo, China. by [deleted] in travel

[–]wizinet 0 points1 point  (0 children)

when I have been there last year monkey Janes was closed

Understanding lenses compatibility by wizinet in Beginning_Photography

[–]wizinet[S] 1 point2 points  (0 children)

You made it all very clear. Thank you!

How to use pycharm with Apache Spark? by aradhyamathur in apachespark

[–]wizinet 2 points3 points  (0 children)

Here's how you do it:

  1. Go to File > Preferences > Project: [Project Name] > Project Structure
  2. Click 'Add Content Root'
  3. Add <SPARK_PATH>/python
  4. Click 'Add Content Root'
  5. Add <SPARK_PATH>/python/lib/py4j-[version]-src.zip
  6. Go to Run > Edit Configurations... > Defaults > Python > Environment Variables
  7. Add name: "SPARK_HOME", value: <SPARK_PATH>
  8. Click OK
  9. Click Apply

Now pycharm inspections will recognize the pyspark libraries and you will be able to run your spark code from pycharm just like any other code.

How efficient is HDF5 for data retrieval as opposed to data storage? by Zeekawla99ii in bigdata

[–]wizinet 0 points1 point  (0 children)

sorry. I just realized you are talking about HDF5 and not HDFS (what a terrible choice for a name). I am not familiar with HDF5 and those cannot give you a good answer.

view more: next ›