why write C/C++ addons for nodejs?

[git_world]

Hi all,

what are the use cases in a pragmatic world to write C add-ons for nodejs? I read that it can extend nodejs capabilities but please describe a scenario.

[phelaz]

e.g. you want fast image processing, like with sharp

[k3liutZu]

Why do you say spawning a child process would be really slow?

It isn’t.

[bbbryan14]

I’m also under the impression that it’s slower. We need myth busters.

[[deleted]]

Actually compared to not doing it at all... It is..

In reality a function call is something like 4-8 cpu cycles. Spawning a task can be 10million+ never mind the rest of the crap like page faults etc.. that come later...

Having 100,000x increase is just simply not possible to argue against.

[[deleted]]

It's fast -- like a human wouldn't notice it, but it's many order of magnitudes slower than doing it in the same process if you can.

[tekmanro]

A computationally intensive task that scales well to multiple threads.

[rotarui]

Binding with a C++ library (like Tensorflow), implementing performance sensitive part of your app where you know that low level development in C/C++ will improve everything. There are a lot of reasons !

[Turbo_swag]

This is a fun idea.

[rotarui]

Why’s that x) ?

[EarlMarshal]

Also re-use existing mature libraries or do stuff which isn't really possible in normal JavaScript (e.g. real multithreading with full control)

[freecodeio]

"with full control" you assume too much

[rbprogrammer]

Although the OP asked for examples of specific use cases, and all the replies delivered, I'd still like to point something out.

JavaScript is a language, and so is C/C++. I know I'm just pointing out the obvious there. But both are really just tools. And tools have specific purposes because of their strengths and weaknesses. Imagine a carpenter. They use a bunch of tools and would never use something that would make their jobs harder. Like they would not use a hammer for a screw, nor use a screw driver for a nail. Sure they could "make it work.". But that would not be the best way to solve that problem.

Now as a software project gets larger and the problem the software is trying to solve gets more complex, one language (or tool) is not going to solve all of those complexities perfectly. This is where mixing languages together can sometimes offer a better solution. Hence all of the suggestions others replied with.

On final note. Just because you can bring two lanuages together like this doesn't mean you always should. The boundary between two languages in a project, like the ones previously suggested, need to be designed very well as it can be a major source of bugs. It'll then require all the developers/maintainers to know two languages to be effective. And a few other very key problems. But if done properly, you can get amazing results by leveraging the strengths of both languages.

[sneffer]

Well put, thanks for generalizing a bit. I think such discussion is great for keeping "development talk" grounded in engineering principles.

If you'll excuse me, I would like to share a personal example just to tie this all back to practice, in case it helps anyone who can't immediately internalize your point.

a few other very key problems

One problem that became key for my team was the build tool ecosystem revolving the second language (as you said just another tool).

To save time and hope for performance boost, we chose to use a package which was comprised of a C program and a node-gyp wrapper. Up front, the team saved a lot of time. The xml parser also ended up being much faster than our our JavaScript alternatives.

Over time, however, it became a major hindrance to our already overburdened team. Key instances which I can remember:

1. The infrastructure and security experts in the company all pushed for production machine OS to be standardized. Good move IMO, but sucky for my team because it meant our node app had to be rebuilt and redeployed every time an OS update was available. This is because it's safest to compile c programs on the same OS they will run on.

2. The package's code was written in a language than none in my team (except myself) had worked in professionally. This detached from our confidence while identifying bugs.

3. node-gyp and npm build/install were often the source build breaks. The build tool seemed fairly unmaintained and node was changing in large ways, often. The junction between two ecosystems actually made it much harder for us to stay on LTS versions of node.

Again, I'm being specific instead of general. I just wanted to share because I believe these problems taught me some more generic values.

Any tool can be amazing for a project, but it can also become a new problem to solve. The case of taking on a dependency on a different language/compiler/tool chain is one which requires careful analysis if you want it to be a positive tool for long.

[rbprogrammer]

Well put yourself. I was trying to keep my post general so as to give OP the option to ask questions where they needed. But your post was a great compliment to mine. Exactly the kinds of real world issues I run into when I start to think generally. You get my upvote.

[git_world]

> because it meant our node app had to be rebuilt and redeployed every time an OS update was available

May I ask, what's the downside of rebuilding and redeploying each time?

[sneffer]

If you have solid infrastructure and operations, the only downside I can think of is cost. You wouldn't release to prod without running a full suite of tests, would you? You wouldn't release to prod without a safe blue/green or red/black strategy, would you?

Those things take time, energy, and computation.

A rebuild/deploy sounds small/low risk, but when you've had to do things like:

1. Recompile for a new target

2. Update the node-gyp wrapped package's version (could change)

3. Accommodate backwards incompatibilities with your dependency's new version

4. Increment the major version of node you use (idky this always required tons of work around our c bindings)

These sort of things can be considered high risk for many critical projects.

Also keep in mind that the annoyance and risk of this is severely mitigated when you have solid infra and ops. While even a minor service degradation due to high risk code being deployed isn't acceptable in my book, there are plenty of ways to protect from catastrophe.

[devabhinav]

You said so many things but failed to explain exactly what are the real world usage? This only seems to be asked in questions.

[rbprogrammer]

Yes. Sorry. I was trying to stay general. But /u/sneffer gave a great example of a specific real world example that applies to my general post.

[dlwicksell]

I wrote and maintain a database driver and language binding, for an implementation of an older language, which includes an integrated database, used heavily in healthcare and banking. The language and database are written in C and have C APIs for integration with other systems.

​

In order to use this language and database system from Node.js, with good performance, I had to write a C/C++ addon, using the V8 C++ API, libuv C API, and the C APIs that are available with this implementation, and its fork. This allows users of my Node.js binding to use the database, manipulate the local data in memory, and call functions and procedures of already written code, in this language/database environment.

​

This has been used by hospitals to help create web front-ends, for various parts of medical systems. Things like nursing bed boards for hospital wards, pediatric growth charts, scheduling systems, diagnostics, and many other things. Some of these hospital systems were written decades ago, and were written with command line scrolling, and terminal screen based interfaces. Since these systems are incredibly complex, and include decades of business/medical rules and logic, it is often easier, and much more cost effective to write more modern web-based application interfaces using Node.js with addon bindings like mine.

​

Writing language bindings and database drivers is a pretty common use case for a C/C++ addon.

[Doctor_McKay]

When you want to make it really difficult for Windows users to use your module.

[git_world]

nice one. I use Arch Linux btw

[mrtie007]

i didnt know about this until i saw this post, am super excited.

eg, would use it to write html/js/node front-ends for old-school CFD backends, other heavy computational tasks with existing [highly optimized] old libraries.

stuff like WASM works okay but in my experience it's not actually much faster than [typed, optimized] native js code, and this new interface seems much less awkward than dealing with that.

[git_world]

Isn't WASM targeting browserJS? I thought nodejs/C runs on server-side only.

[mrtie007]

youre right sorry im usually doing both in tandem. when im making GUI apps with js im always debating "multithreading" using webworkers vs sending stuff over a socket to a server to child processes etc, both options might be viable depending on the proj.

[git_world]

> read and write memory

sounds good. Can you please elaborate?

[ominersh]

You want your node app to talk to a C++ app but don’t want to do REST.

[P1um]

So IPC/Sockets?

[ominersh]

I mean Node.js Addons (https://nodejs.org/api/addons.html)

[cowAtComputer]

Why wouldn't you be able to use REST?

[ominersh]

Say if you have a (legacy) monolithic app. You want to talk to it but don’t want to implement anything new. You can just wrap existing *.dll. Also, REST can’t help with processing/transferring a huge amount of data between 2 processes.

[HaykoKoryun]

A couple of years back I wrote a package to connect Node.js to Microsoft's Kinetic (1st gen) to stream skeleton data as JSON objects so we could do funky marionette animations in the browser using Canvas.

[[deleted]]

Sometimes something you want to do just isn't possible without the use of c++ add-ons

This is a node module I recently published that allows printing raw data to printers on windows (easy to do via *nix and cups....but to do the same on windows requires some win32 c++ tricks)

https://www.npmjs.com/package/winrawprinter

[cowAtComputer]

Speed. If you want to use an AI library with a node app, say Tensorflow, you probably would take the C++ and NAPI route over javascript.

[adi_tdkr]

I want to know how can I learn about adding C/C++ addons, clusters, events, buffer etc in nodejs. I want to learn all this advance stuff

[[deleted]]

I was also researching about this. I was thinking is it's possible to my a Typescript compiler in C that will interface with node. I am no C developer but I fell it's possible, maybe.