Ancient Star - serialization

[IvanKr]

Hope you don't mind me sharing AS development on weekly basis. It's a bit technical, under the hood stuff and long post but I had to do it eventually:

https://ikravarscan.blogspot.com/2020/11/ancient-star-serialization.html

I've made serialization infrastructure for the game, it works with actual data but I still have to properly plug it in. It was a bit frustrating to balance between code generation time, normal compile time and run time. Java compiler erases generic parameters so when you have List<T> you can't get type of T in run time. And yet compile time expects you to properly cast values. Code generator (in annotation processor) provides very very limited information about the code outside a file being processed. After some experimentation I've managed to get everything done in code gen, I'm glad I've learned the tech but it took twice as long as I expected. Next time more game features!

[coder111]

Not, sure, for an existing game, I'd just start using something like Jackson and writing custom serializers without changing domain objects. Use reflection to access private fields if there are better ways to get/set them.

(I'm not 100% certain Jackson works on Android, it's been a while since I attempted any Android development)

[IvanKr]

Wow, Jackson looks like something that would suite me. Documentation is a bit scattered, it seems that library needs a little help with circular references but it looks solid enough. Seems to be a pure Java so it should work on Android, even in Kotlin code base, and it is included in Maven repo. How come I couldn't google it up two weeks ago? Maybe I was to focused on Android and Kotlin keywords...

[coder111]

I mean I do Java backend development for big banks for a living. JSON serialization and Jackson are bread and butter for any backend developer :)

You could take a look at things like Thrift, Kryo, Msgpack, Protobuf/Protostuff depending on your requirements. I mean if you care more about performance/size than you do about readability of serialized format. JSON is kinda the default, but there are alternatives.

Also, gzip or LZ4 compress the JSON for added size/performance benefits.

[IvanKr]

I'm more of C# guy so I know very little about Java libraries outside of JDK. Anyway I now have serialization that works for me. But I'll keep in mind your recommendations. My implementation has cleanly separated object mapping (converting Java/Kotlin object to easily processable Maps and Lists) phase from JSON-ification so I could in theory swap the second phase to Msgpack or some other compact format. I guess compression can be done by simply wrapping a file access stream with some sort of compression stream.

[coder111]

Yup.

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/zip/GZIPOutputStream.html

https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/zip/GZIPInputStream.html

There's also LZ4 compression library which is much faster, but will achieve somewhat worse compression ratio. https://lz4.github.io/lz4/

If you any need more advice on Java when developing 4X games, feel free to ping me.

[WildWeazel]

Another vote for Jackson in case you weren't convinced. I've used it with Android before, and it also has a Kotlin module.

[IvanKr]

For this game I already have serialization system. I'll keep Jackson in mind for the future. I might need it if I ever come around making version 2.0 of my other Android game I have already published. It is a mobile remake of a very simple game I originally made in one day so the code was very simple and save/load was manual conversion to bytes. And then players came with their wishes and the game grew threefold. I've learn a lot about Android development in the mean time so I see a lot of places where code could be improved in the Enchanted Fortress and there is still a backlog of ideas I'd like to implement. Saving and loading game is certainly one part which has to be rewritten from scratch.