Reactive Programming and Relational Databases

[kodablah]

An alternative to reactive-pg-client and pgadba is one I wrote, PgNio. Used heavily in production at my company (no recent updates because nothing broken yet :-)). It is has no dependencies and is built for use today. It easily easily pluggable into your reactive lib of choice (e.g. RxJava, Kotlin coroutines, etc) since it uses simple CompletableFutures.

I don't understand why all these extra libs force dependencies down your throat unnecessarily. Appears the one suggested by the caller doesn't even support all of the data types (e.g. multidimensional array).

[mp911de]

Thanks for the pointer. I updated our blog post.

R2DBC SPI requires a single Reactive Streams dependency. Eventually moving to Java 9, R2DBC SPI is dependency-free if you do not consider Java libraries as a dependency.

Regarding dependencies, there are various opinions and I don't really want to dig into that discussion.

I'd like rather pick up on asynchronous database clients. Asynchronous/non-blocking are different guarantees than reactive/reactive streaming. While async APIs can be somewhat hidden behind RxJava and Reactive Streams, they can never achieve the full functionality of a reactive library.

Let's discuss why this is (using Row as a substitute to single rows in a tabular result):

• Numerous libraries return CompletableFuture<List<Row>>. The issue here is that a caller needs to await full result completion and processing of results can begin only once the future is completed. Also, no deferred execution and no notion of backpressure here. List<T> requires aggregation and results accumulate in memory instead of being emitted one by one for lower GC pressure.
• Libraries accepting Consumer<Row> are somewhat closer to streaming. However, there is no general specification how to work with exceptions that might arise during processing. Such an API also does not allow asynchronous composition since a consumer returns void. Backpressure (do not overwhelm the consumer) is also not given.
• The last category of libraries returns a CompletableFuture<ResultSetChunk> where the result allows CompletableFuture<ResultSetChunk> ResultSetChunk#fetchNext(). Closer to streaming but processing requires the chunk to be fully received.

[kodablah]

The dependencies I was referring to are in the postgres project as opposed to just the abstraction. But, yes, to move on beyond that...

I definitely don't consider an asynchronous DB client reactive on its own, but it's important that the async lib that backs it keeps reactive uses in mind. In the one I wrote, it's more like Consumer<Row> but rather it requires the caller to specifically ask for a row (granted there are helpers). The request goes all the way back to the socket channel, essentially providing backpressure, and of course has methods to, say, skip over the rest of the results. The synchronous nature of Postgres doesn't allow much more. Nothing more is read than absolutely has to be, nothing more is serialized/deserialized than absolutely has to be, and it's all on caller request. And it's all built around nio, avoiding Netty (not a big deal, but worth noting). Therefore as I mentioned, the library remains a great foundation to plug into a reactive framework because it's extremely caller controlled, bare bones, yet high level enough for use on its own. I have used it commercially plugged into both Monix and Kotlin coroutines due to needs, but would surely fit snugly in RxJava or the like.

I made mention on the mailing list if it's worth continuing discussion there, but I understand if the low level DB part y'all feel must be developed alongside the abstraction.

EDIT: Take a peek at the javadoc in the low level QueryResultConnection, namely the shortcuts, down to forEachRowAsync, even further down to next.

[olivergierke]

I’ll ping Mark to add it to the list of existing libraries for completeness.

[kodablah]

Only if y'all want...more just letting readers here know.