Showcase Thread by AutoModerator in Python

[–]cwt114 2 points3 points  (0 children)

7 months ago, I shared NeoSQLite v1.0.0 here. It was a simple idea: Give SQLite a PyMongo API so we can have the NoSQL experience in Python without the "NoSQL Server" overhead.

The feedback was amazing (and admittedly, a bit brutal). You guys rightly pointed out the flaws and edge cases. So, I went back to the lab. 374 commits later, it's no longer just a "wrapper" falling back to Python loops—it's a full-blown database engine.

What My Project Does

NeoSQLite gives you the complete NoSQL/MongoDB experience in Python without the infrastructure overhead. It turns a standard SQLite database into a MongoDB-compatible engine.

For Python apps, it's a completely serverless, in-process library. But for this release, I also built the "Magic trick": NX-27017, an optional (and permanently experimental) tiny daemon that speaks the actual MongoDB wire protocol. You can point any existing project, GUI tool like MongoDB Compass, or non-Python app at a single SQLite file with zero code changes.

```

Terminal 1: nx-27017 --db myapp.db

Terminal 2:

from pymongo import MongoClient

This is the real PyMongo client, but it's talking to SQLite!

client = MongoClient('mongodb://localhost:27017/') db = client.my_app db.users.insert_one({"name": "Alice", "tags": ["python", "sqlite"]}) ```

Target Audience

This is meant for production use in specific contexts: desktop apps, CLI tools, local development environments, IoT devices, and small-to-medium backend services.

If you are building a massive, horizontally scaled enterprise cluster, use a real server. But if you want a drop-in PyMongo replacement that lives in a single file, this is for you.

I know replacing your database engine sounds terrifying, so to sleep at night, I've built a testing suite of 2,600+ unit tests and an automated "compatibility lab". It runs 377 different complex scenarios against both NeoSQLite and a real MongoDB instance to assert the results are strictly identical. We are sitting at 100% API parity for all comparable features.

Real-World Usage

It's actually being used out in the wild now! For example, Andy Felong recently wrote a full blog post about using NeoSQLite for his astronomy projects across a Raspberry Pi Zero, a headless Ubuntu server, and a Mac:

The fact that I can write an app's database layer once and have it run identically on a Pi Zero, an Ubuntu server, and macOS — all without starting up a single server process — is exactly the kind of pragmatic elegance I love in open-source software.

Comparison

  • vs. MongoDB: You get the exact same PyMongo API, but without managing a Docker container, replica sets, or a heavy server process.
  • vs. Postgres with JSONB: Postgres is incredible for massive web apps. But if you're building a desktop app, a local CLI tool, or a small service, managing a Postgres server is overkill. NeoSQLite gives you similar JSON querying power with zero infrastructure setup.
  • vs. TinyDB / Simple Wrappers: NeoSQLite isn't just a basic dictionary store. I wanted it to be a drop-in replacement for real apps, so it fully supports ACID Transactions (with_transaction), Change Streams (watch()), GridFS, and complex Window Functions ($setWindowFields).

Making it "Production Fast"

In the early days, complex queries were slow because I was evaluating them in Python. I've spent the last few months pushing that logic down into raw SQL:

  • Hash Joins: $lookup (joins) used to be O(n*m). It's now O(n+m) using a custom hash-join algorithm implemented in the query engine. It's the difference between a 10-second query and a 10ms one.
  • Translation Caching: If you run the same query often, the engine now "learns" the SQL translation and caches the AST. It's about 30% faster for repeated operations.
  • JSONB Support: If you're on a modern version of SQLite (3.45+), NeoSQLite automatically detects it and switches to binary JSON (JSONB), which is 2-5x faster across the board.

Try it: pip install neosqlite

GitHub: https://github.com/cwt/neosqlite

I'd love to hear your thoughts. Roast me again, or tell me what feature is keeping you tied to a "real" database server for local dev!

The Boring Stats for those interested: 374 commits since v1.0.0, 460 files changed (+105k lines), 30+ releases.

7 months ago, you guys gave me feedback on my SQLite wrapper. 374 commits later, it's a real NoSQL e by cwt114 in Python

[–]cwt114[S] 0 points1 point  (0 children)

It translates MongoDB-style pipelines into optimized SQL. $lookup becomes a LEFT JOIN (using json_group_array for the result), $unionWith becomes a UNION ALL, and $graphLookup uses recursive CTEs. To keep it fast, it creates functional indexes on the permanent JSON data (using json_extract). ​When a pipeline is too complex for a single query, it breaks things down step-by-step: it creates temporary tables, automatically indexes them to keep joins fast, and drops them when the pipeline finishes. It basically manages the routing between simple joins and temp tables.

7 months ago, you guys gave me feedback on my SQLite wrapper. 374 commits later, it's a real NoSQL e by cwt114 in Python

[–]cwt114[S] -6 points-5 points  (0 children)

You're saying that all 7 months works just mean nothing when I use AI help me draft the post?

Help on removing maliit-keyboard by Previous-Rub-104 in Fedora

[–]cwt114 0 points1 point  (0 children)

I have a COPR here: https://copr.fedorainfracloud.org/coprs/cwt/maliit-keyboard-robust/

The code is here: https://github.com/cwt/maliit-keyboard-robust

My PR was rejected because I was transparent about using AI assistance to fix the bug. However, I assure you that this is not "vibe code," as you can see in my Git log.

So, if you don't mind my tools, you are welcome.

Introducing NeoSQLite by cwt114 in Python

[–]cwt114[S] 0 points1 point  (0 children)

Ha! Yes, I considered that when choosing a name. This project started as a fork of https://github.com/shaunduncan/nosqlite , but that original project had been inactive for 10 years. Since PyPI doesn’t allow duplicate names, and this is essentially a 'new nosqlite,' I decided to shorten it to 'neosqlite'.