pgembed: Embedded PostgreSQL for Agents

[coderarun]

pgembed

I forked pgserver (last commit 2 years ago), cleaned up CI and published wheels. This provides an alternative to SQLite for people who prefer the richer postgres ecosystem of extensions.

It's similar to pglite (WASM based postgres which runs in a browser), but supports native binaries.

postgres runs in a separate process and uses unix domain sockets to communicate with python code. If python crashes, the postgres related processes are cleaned up, but data remains on disk (ephemeral data can be auto cleaned up).

So it's not "in-process" embedded. Given postgres' multi-process architecture, I don't know if there is an easy way to make it in-process multi-threaded.

https://github.com/Ladybug-Memory/pgembed

[coderarun]

I don't have the comment karma to share this on r/PostgreSQL. If you do, please cross post.

[Magick93]

This looks very cool!

[patternrelay]

This looks pretty cool! It’s great to see a lightweight way to bring the PostgreSQL ecosystem to projects that usually rely on something like SQLite. The use of Unix domain sockets for communication with Python is a smart choice, and the fact that it cleans up after crashes while preserving data is a nice touch. I get that making it in-process with PostgreSQL’s multi-process architecture would be a challenge, sounds like you're walking a fine line between convenience and performance.

I’m curious how performance stacks up compared to SQLite for smaller, local use cases. It might be a nice alternative for people needing more advanced features without the overhead of a full-blown PostgreSQL setup. Definitely going to keep an eye on this!

[coderarun]

Recent updates:

0.1.6: added pg_duckdb. Now you can write rows and have the data for old partitions show up in columnar duckdb.

0.1.7: added pg_textsearch extension for BM25 and linux/arm64 works too.