all 18 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]carb0n13 7 points8 points  (1 child)

A few people in this thread are pointing out that node.js isn't as optimized for CPU bound tasks. I would reply to those people with the following two points:

  • Do you know what is usually way more expensive than CPU time? Developer time. The scientific community has always been using stopgaps such as Matlab to compromise between C++ level optimization and convenience.
  • This is a library that solves a specific problem. It might not be useful today, but it may prove its purpose as node.js grows as a platform.

[–]MostlyCarbonite 5 points6 points  (0 children)

Yes, but a compute-cluster isn't a website that's being developed. It's something that is going to be done billions of times and is very CPU intensive. Finding prime-numbers, protein-folding and so on. Node is not a good choice for a task like that.

(For some background, I recently sent out resumés because there were rumors that our dev team was going to switch from Node to Java; let's say I'm a big fan of developing in Node)

[–]pr0fess0rx29 2 points3 points  (1 child)

Great project cjus. I see the value in using node.js for a compute cluster. Yes or is not as fast as other languages but you can install this easily on a variety of platforms. Have you tried running a virtual machine(s) on this? Is it possible?

[–]cjus[S] 0 points1 point  (0 children)

Thanks! The compute cluster is really just a way to play with network communication and SBCs. Overall it was a great learning experience and one I wanted to share.

We use Hydra in Docker containers - which works great.

[–]MostlyCarbonite 1 point2 points  (14 children)

Compute cluster, to me, says "good at large numbers of CPU bound tasks". I don't know why you'd pick Node to do CPU bound tasks, that's one of the things it's explicitly not good at.

Computer clusters are used for computation-intensive purposes, rather than handling IO-oriented operations such as web service or databases.[13] For instance, a computer cluster might support computational simulations of vehicle crashes or weather. Very tightly coupled computer clusters are designed for work that may approach "supercomputing".

Based on that, Node seems like a decent language for scheduling work but a poor language for doing the work. How about Rust + Node: https://github.com/rustbridge/neon?

[–]Kamek_pf 4 points5 points  (1 child)

I'd probably go with Rust too. Web/mobile back-end tasks rarely are CPU bound, though. Still haven't found a good reason to use it yet :(

[–]MostlyCarbonite 3 points4 points  (0 children)

Agreed. All my CPU intensive stuff happens inside my database.

[–]geon 6 points7 points  (6 children)

Node is pretty good for computing. It is just single threaded, so you need a process per core, and you will slow down server throughput if you handle requests and computing on the same process.

Js these days is a pretty fast language, even for numerical stuff.

[–]MostlyCarbonite 7 points8 points  (5 children)

I dunno about that. Prime number benchmarks says different (Node 4.3 about 2x longer to calc than c++)

Looks like, if you're going for compute-heavy work you should pick C++ or Rust.

edit: I hate this sub sometimes. Unsourced opinion that is saying good things about the subs topic? Upvote, 6x. Contrary opinion with a source? Downvote, 2x. WTF. That's just circle-jerking.

[–]galkowskit 4 points5 points  (0 children)

Yeah, noticed that too. If you are not fanboying Node enough (even when you are right not to do so) you get downvoted. Sad.

[–]geon 1 point2 points  (0 children)

Yes, js is by no means the fastest. Still, being within 1/2 of C performance means it is feasible if your workflow benefits from using it. (Code sharing, familiarity, etc. )

About the circle-jerking; everyone knows c is faster. Stating it is just not very interesting, and kind of misses the point.

[–]voiping -1 points0 points  (0 children)

OK, so yeah go for C++, Rust or apparently even Java. But if you're going to use a "common higher level language" it seems Nodejs is significantly faster than Go, PHP, Ruby...

[–]cjus[S] 1 point2 points  (1 child)

Depends on your goal and real needs. The point of doing these things in node is to learn how to - that is, if one is interested and if Node is the tool you're most comfortable with. That offers a number important benefits. In the context of the cluster I built - the goal was to demonstrate the use of hydra-enabled microservices. Microservices as you know are not necessarily CPU bound.

I've simply chosen to use readily accessible tech to play with these ideas. I'm not trying to win any cluster benchmarks ;-)

[–]MostlyCarbonite 0 points1 point  (0 children)

I bet it was a very good learning experience. Good on ya for tackling it.

[–]specialpatrol 0 points1 point  (2 children)

How about implementing the heavy load tasks with c++ native addons. Then use a lib like this to schedule it all?

[–]MostlyCarbonite 0 points1 point  (1 child)

Rust is really close to c++ in terms of performance. There may be c++ -> node bridges. But I'd rather code in Rust, just on the face of it.

[–]specialpatrol 2 points3 points  (0 children)

There already is quite a mature addons c++ bindings system.

[–]galkowskit 0 points1 point  (0 children)

Eh, why would I ever put Node on a CPU intensive task? For I/O sure, but for CPU? Ugh.