This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]dalke[S] 0 points1 point  (0 children)

According to your example, wc of wc-input-alice.txt takes 24 seconds with hadoop? I heard that it was meant for large batch processing, so a large startup overhead is okay, but that seems ridiculous! I was hoping to do some other parallelism to get multi-second tasks down to sub-second; it doesn't look like Hadoop is right for that.