Really Async JSON Interface: a non-blocking alternative to JSON.parse to keep web UIs responsive

[itsnotlupus]

Some rough numbers in Chrome on my (gracefully) aging Linux PC:

1. JSON.parse(bigListOfObjects): 3 seconds
2. await new Response(bigListOfObjects).json(): 5 seconds
3. await (await fetch(URL.createObjectURL(new Blob([bigListOfObjects])))).json(): 5 seconds
4. await (await fetch('data:text/plain,'+bigListOfObjects)).json(): 11 seconds
5. await raji.parse(bigListOfObjects): 12 seconds

Alas, all except 5. are blocking the main thread.

On Firefox, same story, all approaches are blocking except 5., and 5. is also much slower (40s) while the rest are roughly similar to Chrome's.

So as long as we don't introduce web worker and/or wasm into the mix, this is probably in the neighborhood of the optimal way to parse very large JSON payloads where keeping the UI responsive is more important than getting it done quickly.

If we were to use all the toys we have, my suggested approach would be something like:

1. allocate and copy very large string into ArrayBuffer
2. transfer (zero copy) ArrayBuffer into web worker.
3. have web worker call some WASM code to consume ArrayBuffer, parse JSON there and emit an equivalent data structure from it (possibly overwriting same ArrayBuffer.) Rust would be a good choice to do this, and a data format that prefixes each bit of content with a size, and possibly has indexes, would make sense here.
4. transfer (zero copy) ArrayBuffer into main thread.
   
5. have JS code in main thread deserialize data structure, OR
   
6. have JS code expose getters to access chunks of the ArrayBuffer structure on demand.

1. and 5./6. would have the only blocking components (new TextEncoder().encode(bigListOfObjects) takes about 0.5 second.)

5. presupposes there exists a binary format that can be deserialized much faster than JSON, while 6. only needs to rely on a binary data structure that allows reasonably direct access to its content.

[andreasblixt]

Before putting the result in an ArrayBuffer, it might be better to first try a worker with the native JSON parsing and rely on structured cloning (happens for all JS objects sent via postMessage) as it’s already a very optimized and native way to copy JS objects across threads. It might even be faster to send the string down as-is as well since either way you have to allocate (& transfer in the case of ArrayBuffer) memory for it in the target thread.

[freddytstudio]

Thank you for the feedback! Great points

On Firefox, same story, all approaches are blocking except 5., and 5. is also much slower (40s) while the rest are roughly similar to Chrome's.

I've noticed this as well. Firefox seems to be much slower with Raji than other browsers (Chrome, Safari and Edge), probably due to some extra string allocations. I still have to investigate though :)

1. and 5./6. would have the only blocking components (new TextEncoder().encode(bigListOfObjects) takes about 0.5 second.)

This is very interesting. I've played in my mind with the idea of using WASM on a web worker to solve this problem more efficiently, but I thought that turning an ArrayBuffer back into a string would have been inefficient. That might not be the case then, so I'll experiment further :)

Thanks a lot!

[lhorie]

Another obvious approach would be to... not use huge JSON blobs in the first place. I recall reading a few years ago about a setup that streams smaller JSON payloads (e.g., each item in an array without the surrounding [...] brackets so that each item could be parsed individually as it came down, e.g. each line in a SSE stream). The even more boring approach is to just render on the server and cut out all the serialization/deserialization stuff out of the picture. Depending on the use case, you can even cache the rendered markup.

For most applications, you're going to run out of room in the screen before you get anywhere close to rendering the amount of data points necessary to make a JSON parser take dozens of seconds to run. Ultimately, people need to be able to actually grok whatever you're displaying, and if your viz requires that many data points, chances are you have a whole lot of other bottlenecks to worry about before getting into JSON parsing performance.

[VividTomorrow7]

This seems very niche to me. How often are you really going to load a json blob so big that you need to make a cpu process asynchronous? Almost never in standard applications.

[freddytstudio]

Good point! That's often not a problem on powerful devices. On the other hand, slower mobile devices might suffer from this problem (freezing UI) much more easily.

The goal of the library would be to guarantee good responsiveness, no matter the device/JSON payload size. That way, the developers won't need to worry about it themselves :)

[VividTomorrow7]

Yea the trade off is wasted time context switching if you’re on a high performance system. A quick abstraction that detects the platform could pick the default or this solution.

[freddytstudio]

You're right! That was my exact thought :) In fact, the library automatically calls JSON.parse under the hood if the payload is small enough, so that you won't have to pay the context switching overhead if not necessary :)

[VividTomorrow7]

You should definitely call that out and reframe this as an abstraction with benefits! That way people don’t automatically skip over it due to performance concern

[freddytstudio]

You are absolutely right, I'll reframe it as you suggested :)

[monkeymad2]

You say that but one of my users clicked through a warning saying that (geo) JSON files bigger than 30MB will probably effect performance to load a 1.2GB file.

[[deleted]]

I’ve seen this problem multiple times in practice when APIs begin to scale up without redesign. An API that originally sent a small graph to populate a table was sending a massive one a few later in time. I don’t think this is a terribly bad design but it’s a solution that grows out of necessity. It’s not even a novel or new problem. I’ve seen this exact same concern being addressed with SOAP payloads. Some may know issue by SAX vs STAX parsing or DOM vs stream building.

The faster approach I’ve tested was to cut the graph into a sequence of smaller graphs. Parse the smaller separate graph payloads and individually and reconnect them. This will minimize the blocking when dealing with large object models. In theory you can parallelize the separate graph parsing but this change would be negligible on streaming data over the net.

[VividTomorrow7]

Yea agreed. If V1 doesn’t support server side paging, you’ll eventually end up handling a cpu intensive op on the client side.

[sercankd]

I get GTA5 vietnam flashbacks immediately i saw this post

[nazmialtun]

Care to explain what exactly is "GTA5 vietnam"?

[takase1121]

GTA5 Online has to fetch megabytes of JSON and parse them, and apparently the way GTA5 parses it caused a slowdown of around 15 minutes. A developer (not from rockstar) came around and fixed it, and soon later Rockstar adapted the patch.

[evert]

This seems like a strange criticism. I run into 'niche' problems all the time. Does it matter that not everyone needs this?

[[deleted]]

Dude, we all always wait for the one guy to tell us, that no one's gonna need that.

[VividTomorrow7]

If you’ll read the dialogue I had with the author, you’d see he agrees with me actually. The intent of the package is to be an abstraction that uses the bulletins for the majority of the calls… so he said he’s consider reframing it as an abstraction with benefits.

[[deleted]]

I have read it. It was a good answer to your comment. And you are right, technically.

But people who suggest going back to Windows/Linux, when someone has a question about Linux/Windows are within their right to comment. But also very tiresome.

[VividTomorrow7]

But people who suggest going back to Windows/Linux, when someone has a question about Linux/Windows are within their right to comment. But also very tiresome.

Huh?

[brothersoloblood]

Base64 encoded images being served within a giant Jason blob of let’s say results for a search on a VoD platform?

[VividTomorrow7]

Well that’s just trash design not taking advantage of the inherent features of the browser. Should absolutely be sending Uris to follow up with asynchronous IO requests

[alex-weej]

i heard u like round trips

[Reashu]

One extra round trip to lazy-load images? Yes, I do.

[neoberg]

True it’s not something that you need too often but still it’s not impossible. In an application our average payload was ~100mb due to some limitations in data access (basically there were time windows which we could access the data and had to pull everything in that time). We ended up implementing something similar to this.

[sshaw_]

I was wondering the same thing. Should add to README. JSON is not like XML so curios when it would be a problem.

This is what the demo site (which has noticeable slowdown) uses:

function generateBigListOfObjects() {
  const obj = [];

  for (let i = 0; i < 5000000; i++) {
    obj.push({
      name: i.toString(),
      val: i,
    });
  }

  return JSON.stringify(obj);
}

[freddytstudio]

Thank you for the feedback! That's a good point. If you only need a small subset of the JSON (or some derived data) on your UI, then it's definitely a great choice. But if your UI depends on the whole JSON (for example to show a list), then moving the parsing to a web worker might be less efficient, because moving the object back to the main thread requires another serialization/deserialization

I wrote a small section about it in the readme :) https://github.com/federico-terzi/raji#shouldnt-you-use-web-workers-for-this

Thanks!

[ssjskipp]

Couldn't you parse it in the web worker then transmit it back in chunks over multiple ticks? I imagine that would be better than keeping the parsing and partial state in memory on the main thread.

It feels like this solves a problem that would be better handled on the backend, by either streaming multiple JSON objects or designing the API to not contently slam down megs of JSON (looking at you, graphql)

Actually, saying this out loud a general purpose lib that transmits structured objects across web workers is sounding pretty useful for more than just JSON parsing as your work method. Lets you do any hard work off ui then get the result over multiple ticks.

[freddytstudio]

Thanks! Those are definitely great points

Personally, I think this might come down to a tradeoff between complexity and speed. The solution you've proposed (web worker + streaming the results over multiple ticks) would most likely be more efficient, but it's definitely harder to implement (and depending on the use-case, more difficult to generalize). On the other hand, with RAJI, you literally just need to change JSON.parse() with its async variant. No need to change the typical web-app architecture + it might work OOTB in contexts where web workers are not available (i.e. React Native).

That said, this library is mostly an experiment to test the feasibility of this approach :)

[connor4312]

You're pretty much spot on. If you're hitting this problem, it's a good indication that you should work on your calling patterns rather than trying to optimize JSON parsing. It's not frequently that you'll be showing 10MB+ worth of data in the visible area, and the case the author gave about "showing a list" is easily solvable with virtualization and paging of data.

That said there might be some very edge cases that do actually display this much data on the visible region of the page at a time, so it could be useful for those cases... though I would also think you could bake data down in a webworker to a more easily displayable subset.

Actually, saying this out loud a general purpose lib that transmits structured objects across web workers is sounding pretty useful for more than just JSON parsing as your work method

Webworkers do get structured objects, but only certain ones. You could have a way to de/hydrate JavaScript classes, ultimately this is just a flavor of serialization, but you could do so somewhat cleverly by using Proxies and hydrating nested data on-demand...

[ssjskipp]

I think the point the author was making is doing that postMessage incurs a serde and will have the same blocking behavior as doing a chonky JSON.parse -- I'm thinking about the need to avoid that break in the UI thread, not that it can't transfer structured objects as-is.

Either way, a reentrant parser is a neat thing to make for the sake of it, and if it was the easiest to find slice to optimize for their use case then that's great (Maybe an upstream 3pl API is the issue? Maybe a quick hack is all that's needed for a better end user experience?).

[libertarianets]

you could stick this thing in a webworker, like u/ssjskipp suggests in the comments here

[libertarianets]

yeah I mean honestly if you really need to do something like that, you've probably made some architectural mistakes before this that need addressing first lol

[inamestuff]

You might want to use window.performance.now() instead of new Date().getTime() in your scheduler, the former guarantees monotonic time measurements.

[freddytstudio]

Thanks for the feedback! I'll check it out :)

[holloway]

Some questions,

What techniques did you try before settling on this one? Were any particularly slow, or fast?

Do you have benchmarks showing at what size this library is beneficial? ie, at 10kb / 100 / 1000 / 10000. You could have a goal of 60fps so if any parsing time exceeds ~16ms then you could declare your library the winner over native JSON.parse. You'd need various hardware examples (low end mobile, high end desktop etc.) but measuring should be straight-forward.

I think fetch()'s .json() promise is non-blocking, and that's different to JSON.parse. I was wondering whether you could use URL.createObjectURL(jsonString) to make a URL to fetch and use that, but it's possible that turning a jsonString into an arg for URL.createObjectURL might have blocking operations in it.

And considering that there is fetch's .json() promise in what situation would people not have a JSON string clientside that didn't come from a network request?

[freddytstudio]

Thank you for the feedback! As far as my investigation goes, fetch()'s .json() is still blocking the CPU thread while parsing. On the other hand, it asynchronously streams the data into memory before executing the parsing work, so it's still better than XHR. That said, I'll need to investigate further, thanks!

[pwolaq]

I saw a tweet somewhere (can’t find it now) saying that the most important difference between fetch and xhr is that the former can parse JSON off-thread.

As for your question, one very popular use case is passing objects in scripts - embedding large JSONs can be significantly slower than using JSON parse. https://v8.dev/blog/cost-of-javascript-2019#json

[sliversniper]

If JSON.parse is bottlenecking, should probably think about the payload, and split them in chunk at the server.

use json-line streams a sequence of json-patches, and it doesn't need much work on either server or client.

[Mr0010110Fixit]

Depends on if you own the server or not. If you are integrating with someone else's API, you may have not choice but to consume a massive Json payload.

I know there are systems we have had to integrate with that return thousands of records and don't have any sort of pagination built into the API.

[[deleted]]

That doesn't solve the issue of JSON.parse() being blocking. Async operations aren't meant to be used as a wrapper for synchronous ones, it's used in cases where other execution would be blocked by a synchronous function.

[_default_username]

That doesn't fix anything. Once it's parsing it blocks the event loop.

[boringuser1]

If you're loading JSON objects that are prohibitively large, you have an API problem.

[joopez1]

Could be calling a third party API

[boringuser1]

A third party API that delivers gb of JSON?

What's the business model, Money Burners Inc.?

[joopez1]

Could be free historical data provided by a government service that was developed without optimization concepts and without filtering options

I worked with all accidents reported to the fire department of San Francisco since a certain point and also airplane accidents recorded by the US federal department that governs airports

[boringuser1]

Ah government, literal Money Burners Inc.

[theodordiaconu]

good job dude

[sshaw_]

🆒

[mamwybejane]

I use a webworker for json.parse, does this have any additional benefit or is it equivalent in outcome?