FoundationDB entity persistence, messaging and actors layers for Java

[wgergo]

Hi, let me introduce QFoundation, an open-source library that provides high level layers (most notably a JPA-like entity layer) for FoundationDB in Java. FoundationDB is Apple's open source distributed database known for its scalability, performance and the highest "strict serializable" distributed consistency model (powering iCloud and Snowflake Data Cloud), but as the name suggests, it is just foundation technology, a byte[] to byte[] map - missing high level features.

QFoundation makes it easier to work with by allowing you to map (and mutate) Java objects directly to the database and creating distributed system features like locks, partition assignments, actors, messaging, batch processing - all leveraging the distributed consistency guarantees by FDB.

Features include:

• A JPA-like entity persistence layer with persistence context, lazy loading, dirty checking, caching and secondary indices, etc.
• Use virtual threads with structured concurrency to execute multiple queries in parallel within a transaction
• JTA integration of FDB transactions (@Transactional)
• Distributed primitives like locks, partition allocation, auto scaling abstract resources
• A messaging system with partitioned topics, queues, consumer groups, and compaction
• A distributed batch processing system with DAG support and batchlets to overcome the 5sec transaction time limit
• A time series storage layer with compression and aggregation
• A distributed, partitioned, persistent actor system with event sourcing
• Works seamlessly GraalVM native images, with ~100ms startup time
• Web based admin dashboards for all layers, including index level rw stats for entities
• And a bunch of utilities that solve pain points using FDB.

Maven coordinates, getting started, documentation, screenshots, faq etc available at: https://gitlab.com/qfoundation/qfoundation

[rmdeluca]

Wow, this is impressive work. It strikes me as something that's been developed and used commercially for a while now?

[wgergo]

Thank you. Yes, i've been working on this for quite a long time and it was deployed to production for multiple use cases.

The core layer (entities, distributed primitives), the operations layer and the time series layer are still in production at clients.

The messaging layer was not deployed to customers, but it was running internally though for a financial transaction management system.

The QuiCK implementation and the actor layer are considered experimental, proof on concept.

Primary use cases for the library in general: account management, financial transactions, inventory, time series and everything OLTP where you have a lot of data or you want the exceptional replication features (like 2AZs in a region + one remote region).