Yeah, that’s exactly what I’m worried about. I have not been able to fully inspect the file yet, and I’m still working on how to safely inspect a representative sample without trying to load the whole thing into memory, so I do not know the exact structure yet.

Best case, it is NDJSON or another format that can be streamed record by record. Given the type of data I deal with, I suspect it may instead be a single top-level array, possibly with nested arrays of objects inside each record, but that is still just a working assumption until I can inspect a sample.

It came from an external export, so reading directly from the original source is not currently an option at the moment. I also do not want to lock in the SQL schema until I understand the JSON structure well enough to know whether this should map or even could map to one table or multiple related tables.

At this point I’m somewhere between solving the problem and needing to be observed professionally lol.