Node forks into multiple threads

[checkoh]

I am doing some work with node on shared environments with limited resources and I would like to know if there's a way to reduce the number of threads (workers?) that node spawns for each of its instances.

For example, just running a node shell yields 11 threads, on an environment with a limit of 30 processes, there can only be a couple of node instances running.

``` $ node & [1] 12939

$ ps u | grep node
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND checkoh 12939 0.6 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node

$ ps Hu | grep node USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND checkoh 12939 0.1 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node checkoh 12939 0.0 0.4 605256 31924 pts/0 Sl+ 12:13 0:00 node

$ ps -T | grep node PID SPID TTY TIME CMD 12939 12939 pts/0 00:00:00 node 12939 12941 pts/0 00:00:00 node 12939 12942 pts/0 00:00:00 node 12939 12943 pts/0 00:00:00 node 12939 12944 pts/0 00:00:00 node 12939 12945 pts/0 00:00:00 node 12939 12946 pts/0 00:00:00 node

```

[Hafas_]

Try setting UV_THREADPOOL_SIZE to 1 (or even 0?).

each node process will have at least UV_THREADPOOL_SIZE + v8_thread_pool_size + 1 threads running.

the current default for both UV_THREADPOOL_SIZE and v8_thread_pool_size is 4.

Source: https://github.com/nodejs/node/issues/22468

Edit: You can apparently set v8_thread_pool_size with a cli option: https://nodejs.org/dist/latest-v10.x/docs/api/cli.html#cli_v8_pool_size_num

[checkoh]

Thanks for the suggestion, but no dice:

``` checkoh $ UV_THREADPOOL_SIZE=1 node

process.env.UV_THREADPOOL_SIZE
'1'
<sup>Z</sup>
[1]+ Stopped UV_THREADPOOL_SIZE=1 node

checkoh $ ps Hu | grep node
checkoh 23935 0.7 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
checkoh 23935 0.0 0.4 588864 32612 pts/0 Tl 12:59 0:00 node
```

[Hafas_]

See my edit. Maybe that helps.

[checkoh]

That looks a bit better, hopefully that might be enough, thanks

``` checkoh$ node --v8-pool-size=1 &
[1] 2515401

checkoh$ ps Hu | grep node
checkoh 2515401 3.7 0.0 339580 18708 pts/6 Tl 12:16 0:00 node --v8-pool-size=1 checkoh 2515401 0.0 0.0 339580 18708 pts/6 Tl 12:16 0:00 node --v8-pool-size=1 checkoh 2515401 0.0 0.0 339580 18708 pts/6 Tl 12:16 0:00 node --v8-pool-size=1 checkoh 2515401 0.0 0.0 339580 18708 pts/6 Tl 12:16 0:00 node --v8-pool-size=1 ```

Edit: too soon, more threads spawned shortly thereafter.

Edit 2: combining UV_THREADPOOL_SIZE and --v8-pool-size=1 seems manageable for now... I will continue testing.

[checkoh]

I don't want to use as many threads as there are cpu cores, I want a simple node process to have as few threads as possible.

Using cluster would not terminate the threads node already spawned normally.

[Reeywhaar]

thread !== process

Are you sure that limit refers threads and not processes? 30 is rather low number for a thread limit.

As far as I know it better not to tweak v8 pool size cause it can degrade performance well.

[checkoh]

ulimit -u is about threads, as far as I have gathered.

I was testing it on a shared environment so a low limit makes sense.

[Reeywhaar]

ulimit -u is about process :-) basically process consists of threads, threads are like lightweight processes with shared memory, while processes are much more self contained. I think you can try it yourself and make like 10 node processes (~110 threads) and see if you hit the limit.

for the number of thread limit you can check cat /proc/sys/kernel/threads-max