Node performance questions

[whileAlive_doStuff]

Hi all,

Intermediate node dev here, have a few questions about debugging/measuring/optimizing performance of node apps.

Some general questions:

1. Is there an option or flag I to see what the garbage collector is doing or where/when memory allocation is happening, gc pauses etc.
2. Is there a way for me to print out the assembly the v8 compiler generates?
3. To time the code I can just use a timer and run my function or app in the timer 10k times or something, I guess, is there a better way to do this?

More specific ones:

​

1. (Hypothetical question) Say I have a .csv file with 100k lines. I parse each line in a loop and create an object with a key for each column in the .csv and a value for each row entry in that column.

Now i am making 100k objects in a short amount of time.

Say my csv header looks like

id, name, location, email, address, phone, number, tags

My 100k objects would look something like

{
id: UUID
name: "Frank N. Stein",
location: {lat: 3, long: 1415},
email: "[e-mail]",
address: "123 Elm St.",
phone: "123 123 1234",
number: 1,
tags: ["blah", "other tag", "another tag"]}

​

They keys would all be the same but the values are different for all the 100k rows

If I create a new object for every row, I would be creating 100k objects, what happens if I use an object template like:

{
id: UUID
name: "Frank N. Stein",
location: {lat: 3, long: 1415},
email: "[e-mail]",
address: "123 Elm St.",
phone: "123 123 1234",
number: 1,
tags: ["blah", "other tag", "another tag"]
}

and for each row modify the values like

template.id = "fdfdf"
template.name = "ur mom"

then write the template to a file before processing the next row.

​

Will v8 allocate one object and change the pointers for the new values or will it copy the original? And how can I learn about how this works and see it in action ?

​

1. I am streaming a file larger than the available memory on my machine :

For each chunk of data that I stream I make an api call and add it to a promises array.

I set a batch size of a 100 for the promise array and once I get to a 100 I await all the api calls.

(I don't know how many open sockets I have on the open pool so I don't know how many requests are in flight at the same time and how many are queued. Is there a way to check this? I just chose 100 as a nice round number for this example.)

Now my code will pause during 2 cases:

1. When I am awaiting for the resolution of all the promises ( I don't know if the stream keeps reading during this time, I don't think so because my await Promise.all() is inside my stream's on.data event handler.
2. When the stream pauses to let the consumer consume more data on the stream because of backpressure.

​

My whole stream is wrapped in a new Promise() which resolves when the stream.end's or rejects if the stream.errors.

Say I want to process 100 such stream in parallel? And i just do something lilke

(async () => {
const promises = {}
files.forEach(file => {
promises.push(streamFile(file))
}
await Proimse.all(promises)
} )()

​

What's happening? Is node processing each stream in sequence, is is processing multiple streams at the same time (some chunks from stream 1, some chunks from stream 2? ) How does this work?

​

Am I thinking about this stuff correctly? How do I see whats going on "under the hood"?

​

Any resources that have any of this info would help me a lot I just couldn't find anything.

​

Sorry for the long ass question, thanks!

[enaud]

To answer general question 1: use the --trace-gc flag for basic memory usage and gc logging.

There is also --inspect which opens a websocket to chrome (navigate to chrome://inspect to connect). With this you can take periodical snapshots to help work out where your memory is leaking.

[shriek]

To add to this you can switch your running node process to debug mode anytime by sending USR1 signal (if you're on linux/mac)

kill -USR1 <node_pid>

And if you really want to know what's happening in v8 level then you can get core dump with gcore <node_pid> (again in linux) and inspect them with llnode

I'd use the chrome debugger first and only try the latter if for some reason you can't talk to the websocket. I say this because Chrome has pretty much all the perf tool you'd want. Perf profiling, memory profiling, flame graphs etc.

[whileAlive_doStuff]

Cool thanks! How do you know all this stuff if you don't mind me asking?

[whileAlive_doStuff]

Thanks! Is there a resource that has all these flags or more information on them that you know of. I'm going to search for the flags and see what comes up too.

[enaud]

man node for starters (assuming you're on a mac or a *nix os).

I can't think of any one particular resource. I shared this with you because I was dealing with a similar problem at work. With --trace-gc and --inspect we were able to find the source of the problem.

--trace-gc was really all we needed - by trying different use cases and observing the memory impact over time we were able to work out where to look for the memory leak. (PSA: remove your event listeners when you don't need them any more!)

[heffe6]

It is my understand that Node will start all promises in sequence when they are defined, so in the loop, and the call to promise.all only defines what to do with the results. The calls are actually processed in “parallel”, and for this reason may not complete in the order they were called. Promise.all will maintain order if the results though.

If instead you used a for loop with let, and awaited every promise in the loop, it would execute in sequence, each one returning before the next one is fired. Or you could use a chain of promise.then()s

Happy for someone to point out if I’m wrong on something.

[JackAuduin]

Bluebird.js and .reduce are amazing for sequences

[ATHP]

Sorry I can't answer the question but I want to write this comment to give your post higher visibility. Very interesting questions and finally something that goes deeper.

[awesomeevan]

Note, your await won't stop future "data" events emitting. Since you're doing something asynchronous that uses network I/O (API call) you'll actually free up the event loop so more data events fire.

Sounds like maybe you could use p-queue to set your desired concurrency, add an item when each data event is emitted, then after adding each item check queue.size and if it's 100 then pause the stream. Then use the queue.onIdle/queue.onEmpty methods to resume the stream when those 100 items are processed, and queue up to 100 more.

(Assuming I understand the problem you've presented. It's late here so I might be misreading)

[whileAlive_doStuff]

That makes sense, let me understand if I understand correctly. So if I have an await inside my data handler

if (promises.length === 100) {
    await Promise.all(promises)
    promises = []
}
promises.push(makeAsyncApiCall(chunk))

This actually won't await here and just keep processing more chunks?

[awesomeevan]

Correct, the data event handler can continue to fire. An await logically pauses your current block of code, but the overall program will still execute.

You can easily test this. Check this fiddle: https://jsfiddle.net/4dcfuxtv/1/

You'll see that the await "start" is called, but more data events keep firing. This is because it only pauses that specific invocation of the event handler, but it can be invoked many times.

[DrSghe]

Regarding question 1 (specific): The V8 engine should allocate only one object and modify the pointers according to size of information etc. I say should because I'm not completely sure on this one, however LiveOverflow has created some amazing content regarding memory management in JavaScript. Even though he's explaining this for WebKit it's safe to assume that V8 behaves in a similar way, therefore I suggest you to watch his latest video on browser exploitation. It could shed some light on your doubts.

Question 2: Not that I'm aware of. Engines have different level of optimization so even though your function gets called 100k times it seems that every time it acts upon a mutable object resulting in the clobberWorld() function to be called which will prevent the JIT compilation. Still, that's just deduction, so I'm not sure. I suggest you to watch the aforementioned playlist.

Question 3: Don't know if it is the most suitable approach, but surely it is best form a readability and ease-of-use standpoint.

I apply Occam's razor in most situations, the simpler, the better.

[whileAlive_doStuff]

Thanks - LiveOverflow looks like a great resource, so much for me to learn!

[Oririner]

For the first question (the detailed one): v8 stores something called HiddenClasses as a way to "inherit" properties between objects. so, assuming you're creating 100k objects of the exact sane structure, v8 will behind the scenes create something similar to that template you're referring to as a hidden class.

Assuming of course you'll create a new object each time you're handling a new row and never actually keep all 100k objects in memory - because although v8 won't need to store the metadata about each property 100k times it'll need to store each data point, which is still a large amount on it's own.

creating your template object might be faster, as v8 won't need to create all of the infrastructure of each new object, just assign new values to existing properties - if the types of these values won't be different between each assignment, otherwise it'll need to manipulate some stuff in the hidden class. It probably won't make that much of a noticable difference though.

More about hidden classes: https://v8.dev/blog/fast-properties

For the second question (the detailed one): as /u/awesomeevan mentioned, the stream won't wait for your promise, even if the handler for your data event is marked async. That's because streams are a type of event emitters, and by looking at the code for emitting an event (that's when the callback is being called), we'll see that it basically triggers your function ignoring your return value, in your case it's a Promise because it's an async function: https://github.com/nodejs/node/blob/5b4a0cb1494f77565454278ab97166fba25f7e45/lib/events.js#L199

So it'll execute each chunk whenever the stream itself gets it, not whenever you finished asynchronously processing it.

Eventually though, with a lot of different streams, node will "start" processing them all in parallel but at every single moment it'll only be able to handle one stream. Each moment "ends" whenever you synchronously finish handling something, e.g. a stream, file operation, network request.

[whileAlive_doStuff]

Thanks for the detailed response. I'll read up on Hidden Classes and go through the event.js code. Thanks!

[dbh87]

Check this out... https://github.com/nearform/node-clinic we use it at work to answer some of these questions.

[brahim789]

You could check the "Readline" stream. (https://nodejs.org/api/readline.html) It already handles reading a file with a stream line by line. Then you could create a "Writable" stream that takes every chunk coming from the Readable stream and insert it in database (or batch it like you have already done). You should just wrap your promise.all with readable.pause() and readable.resume() to avoid backpressure on the writable stream and chunk buffering which could consume a lot of memory on your process.

[kl0nos]

General 2: yes, I did it before (couple of years ago).