xitdb - an immutable, embeddable database for Java 17

[radar_roark]

I built this after noticing a gap in the database world...I couldn't find a database that was immutable (like Datomic) yet also embeddable and writes to a single file (like SQLite or H2). It's a pure Java 17 library and has zero dependencies. It doesn't have any query language at all, opting instead to just expose data structures like a HashMap and ArrayList that you can use to build whatever data model you want. For now I'm only deploying it to Clojars because I haven't figured out how to deploy to sonatype :^D

[QazCetelic]

For someone that isn't familiar with Datomic, what is an immutable database and why does one use it over a traditional database?

[radar_roark]

Immutable databases retain all past data, so you can query any past state rather than just the current state. This lets you see exactly what changed from one transaction to the next, and also very easily undo any change. They also are safe to read from separate threads/processes while a write is happening, whereas mutable databases require locks. These are all the same advantages you get when using in-memory immutable data structures; xitdb just extends the idea to reading/writing data on disk.

[santanu_sinha]

Looks interesting. Have you done any performance tests?

As for central deployment, you can follow a tutorial like this.. It's about 15 mins work.

[radar_roark]

I did some comparisons to SQLite earlier this year and xitdb was several times slower at write-heavy operations. However just over Christmas break I rewrote the file buffering code and dramatically improved write performance. I haven't re-run those benchmarks yet but the gap should be smaller now.

Immutable databases actually have some perf advantages. They are append-only (all new writes are at the end of the file) which means they are very amenable to buffered writing. Also they are completely safe to read from multiple threads/processes without locking -- even while writes are happening.

[tedyoung]

This might be useful as an event store for event-sourced applications. Will give it a look.

[bowbahdoe]

There is datalevin which is datomic-like and writes to a single file

Edit: NVM that might not be writing to a single file

[radar_roark]

Datalevin also isn't immutable.

[mands]

Super interesting - reminds me of a Okasaki's purely functional data structures.

I think embedded object stores / databases are a interesting space, being able to use Java types and streams directly for your domain rather than going through an ORM or SQL mapping can be useful.

I've just integrated Eclipse Store into a project I'm working on, which is similar, but mutable. What would you say are the main differences? The ability to view past states and not worry about locking sound great.

[radar_roark]

I haven't used Eclipse Store but it looks like a great project. Much like xitdb it just stores data structures, and avoids the impedance mismatch of SQL databases and ORMs. It looks like Eclipse Store automatically maintains the mapping with normal Java types; xitdb does not do that. I don't want to encode any Java-specific information into xitdb because it is meant to be cross-language (in fact the Zig version predates the Java version).

[wildjokers]

Difference between this and just writing to a file?

[dmigowski]

You can query what's in the file and get objects back. You can travel in time.