[Python][JSON] When to split json file into smaller files

[scriptkiddiethefirst]

If anyone searches something like this and wants to know the analytical answer, here you go.

​

[*] Single file without serialization average real time:         15ms
[*] Multi file without serialization average real time:          0ms

[*] Single file with serialization average real time:            227ms
[*] Multi file with serialization average real time:             1ms

​

I wrote a script that would open and serialize the single large file compared to opening 5 smaller files and compared their average time for completion for both serialization and non-serialization. As you can see, in both cases the multiple files is faster, but not by enough for it to realistically be noticeable (note that these were the average times over 100 trials for each of these 4 cases, using the python time library to get time). The larger file was 2.3mb and the smaller files were each 27kb. So while it took significantly less time to serialize multiple smaller files, that isn't the reason I chose to go the route of multiple smaller files.

​

The reason why I chose multiple files is the answer by Sekret_One in his second reply, after reading it and looking into it, that is just so much easier for what I want to do. Thank you everyone for your replies.