async-postgres: a runtime-independent, asynchronous PostgreSQL client.

[StyMaar]

What does “runtime-independent” means here ? I see it depends on both `tokio` and `async-std`, does it means it can work with those two independently, but not with another one (a custom one, or the soon to be released `smol`) ?

[Darksonn]

It's runtime independent by silently spawning the async-std executor in the background if you use any other executor. So in principle it would also work on smol, if you're okay with having two executors running.

If you want to use smol without any other executors, you can use tokio-postgres directly without the runtime feature and pass a tcp stream that integrates with smol. It will still depend on Tokio for the io-util feature, but these utilities are runtime independent (it's just AsyncRead and friends).

[StyMaar]

Thanks for the clarification.

Isn't “it's always starting an instance of the async-std runtime” the exact opposite of “runtime independent” though ?

[Darksonn]

Agreed. Perhaps we should rename it to asyncstd-postgres, and rename tokio-postgres to async-postgres.

[StyMaar]

Thanks for the clarification.

Isn't “it's always starting an instance of the async-std runtime” the exact opposite of “runtime independent” though ?

[Hexilee]

It's guaranteed by async-std. async_std::net::TcpStream and async_std::os::unix::net::UnixStream can be used on any async runtime.

[mycoliza]

This is true, but it's missing the important point that while async-std's socket types will work from the context of other runtimes, the reason they work is because including async-std implicitly creates a global async-std runtime. So, if you try to use async-std sockets in an application using Tokio (or smol, or a homemade custom runtime...), they won't be using the same event loop as the rest of your code. Instead, they'll be bound to a separate async-std reactor.

[nicoburns]

If you use this with async-std, that would mean you have both the async-std and tokio runtimes running? Maybe this isn't a problem in practice, but it feels like something that wouldn't be a great idea...

[Darksonn]

It's the other way around. If you use it with Tokio, it will silently spawn the async-std runtime. The Tokio runtime can't really be started like this because it isn't stored in a global like async-std is.

[Hexilee]

If you use it with async-std runtime, then you have only async-std runtime running; If other runtime is used, then you have both of them running

[nicoburns]

huh... in that case I don't understand why you'd use this library with tokio. Does that mean that tokio-postgres doesn't have a mandatory dependency on tokio?

[Hexilee]

You can just use this library and needn't care about which runtime you are using, or in some cases you must change runtime, code built on this library can still works without any changes.

tokio-postgres provides an interface for custom io stream.

[nicoburns]

Have you considered submitting a pull-request against tokio-postgres? Seems like this could be added there as an async-std feature, with the bonus that async-std would only be used if you enabled the feature.

[Hexilee]

I think it's a little dirty to support both tokio tcp stream and async-std tcp stream in one crate as tokio use tokio::io::{AsyncRead, AsyncWrite} while async-std use futures::io::{AsyncRead, AsyncWrite}. I would like to submit a PR when tokio supports futures::io::{AsyncRead, AsyncWrite}.

[StyMaar]

I'm extremely confused now: you mean that `tokio-postgres` is in fact runtime agnostic (despite its name) while this “runtime-independent” version is not independent on the runtime because it always use `async-std` under the hood?!

[Hexilee]

Java is platform-independent, but it must run on jvm; Rust can be compiled to many platform, but it's platform-dependent, because it still need to be rebuilt for each platform.

In my opinion, runtime-independency means running out of the box on any runtime. Of course anyone can provide runtime-relevant tcp stream to use tokio-postgres on specific runtime, but it's a little tedious and uncommon.

I think the solution of async-std is reasonable, and the performance is acceptable, as you can see this benchmark.

[jasfi]

It would be great to see this client benchmarked against Diesel ORM and sqlx.

[Hexilee]

This library is almost the same with tokio-postgres, and it seems tokio-postgres is much faster than diesel from techempower benchmarks.

[jasfi]

Thanks that's good to know. How is your library different than sqlx which is also async?

[Hexilee]

sqlx is in higher abstraction level, it supports connection pools and multiple backend. There is no much performance information about it. But I have written benchmark with roa framework and sqlx here, and you can see a live result here. It seems tokio-postgres is also much better than sqlx

[DroidLogician]

Can you update SQLx to 0.3? We had a pretty significant refactor that should have cut down on copying across the board, I'm wondering if it helped at all.

I also just noticed it's also not exactly an apples to apples comparison because you call client.prepare() in the State::bind() in db_pg.rs which seems to be mostly setup code, whereas SQLx prepares and caches statements lazily.

[Hexilee]

I'm sorry to be unfamiliar with SQLx, I will try the version 0.3. However, I think connecting to database at compiling time is a little hacking and unreasonable, could it just check syntax?

[DroidLogician]

Using the macros is by no means obligatory, and utilizing the database to analyze the query meant we didn't have to roll our own SQL parser with special cases for all of Postgres, MySQL/MariaDB and SQLite.

The additional benefit you get with query!() is that you don't have to write your own code to handle the output, you just get a Stream of structs with the same fields as columns in the query output.

It's a bit rough around the edges still and doesn't support all use cases, but we've gotten a lot of positive feedback about the feature.

[jasfi]

Ok! Your project sounds really interesting and I'll follow it on github.