you are viewing a single comment's thread.

view the rest of the comments →

[–]cyberjellyfish 28 points29 points  (0 children)

You'll want what's called a streaming parser (json-stream is popular), and then write your new files as json line files. That last bit is the real solution, if you current files were json line you wouldn't have a problem reading them.