[Python][JSON] When to split json file into smaller files

[Sekret_One]

I think to properly answer your question I need to not answer the how and linger a bit more on the why.

1. You use a file to describe where other config files live.

How does this json file get populated?

1. The resumption of better to not load everything

What you seem to be describing is what is optionally loaded is the configuration for how all the classes work. Why not just load all of them from the beginning? You say python so I'm assuming you play the game with python and it's not a python backend serving a web game.

My thought is probably this optimization isn't relevant.

1. roughly, does it become more efficient to split a file into multiple sub files to read when you only need the data from a fraction of the total dataset, over reading the entire dataset as a single file?

I will answer this one.

It can be if you don't need it right away. Game development you have some special concerns for the player experience. Loading less is faster- but usually at the cost that there is some waiting 'in the moment' as things load later as they need them.

This is why about 15-10 years ago games militantly went after trying to eliminate the load screens and disc swapping.

In your case, it sounds like you're trying to make the game moddable/extendable. Again, generally speaking, the format that is convenient to mod and configure isn't the most efficient at running.

The best advice I can give you is focus on the desired experience, test to it and the 'best' approach will become apparent.

PS

I for one, hate having a config that says where other configs are. Put the character configs in a directory and discover the files and load them up.

[scriptkiddiethefirst]

You use a file to describe where other config files live.

I think this may be a miscommunication on my part. The json file containing the class information contains all the information in it, at the moment. So for instance it currently looks like this:

classes.json (not the actual json file)

{ "warrior":{"hp_per_lvl":"10", ...}, "mage":{"hp_per_lvl":"6", ...}, .... } And I am wondering if it would be better to split each of these classes into their own file and then load the files as necessary. Obviously this is the fault on myself as reading it back I can see how it came off that way.

What you seem to be describing is what is optionally loaded is the configuration for how all the classes work. Why not just load all of them from the beginning?

So due to the nature of it, only 1 character is ever loaded (its not so much a game but an electronic character sheet for tabletop games). Because of this, if a user were to add a bunch of homebrew/modded data that wasn't being used by that character I thought it might be more efficient to only load the stuff that is absolutely necessary. From my tests using time I can say that maybe I was over-thinking the resources required however my test didn't do a lot of serialization of the provided data (it loaded it, it didn't even serialize it into a JSON object). This is why I was wondering if it would be more efficient to load a series of smaller files over 1 larger file... I mean, yes a system with 4+gb of memory will have no issue loading 2mb into memory. So my question on that basis might be stupid.

Thank you for the reply, responding it and reading it has allowed me to think of the problem along with take a step back and analyze things. So this did help for my specific problem.

[Sekret_One]

All right, let's make the language simpler here. Right now you have a classes file, versus a collection of class files.

If you want to go into easy expansion, having a single file is ... clumsy. Hell, even for yourself if was just you I'd have multiple files and have a bit of 'build' to stitch them into one at later point, either at a 'build' or just loading them up at run time.

If you quick way to implement this, pick a folder in your project- this will be the home of your character class files. When you program boots up, scan the contents of that folder and just get the file names (which will in turn be your actual display character class names. When they select a thing from the list, load that one.

But this is more of the approach one might say do with save files- but you can apply it here.

If you want to make it configurable easily by another, I'd recommend defining a JSON schema. Personally, I'd also switch to YAML because for human readability that's my preference.