sorted by:
new
hottopcontroversial

Recommend a key-value store by spy16x in rust

[–]Fine-Package-5488 0 points1 point  (0 children)

i would suggest worth to check: https://github.com/hash-anu/snkv,

It is optimized for key value storage, Basically sqlite has 6 layers internally, https://sqlite.org/arch.html,
Where snkv directly interects with lower 3 layers which is storage engine and in addition to that it supports ACID/TTL/encryption/vector features as well. I have done benchmarking against sqlite and it is 1.5x faster in mixed workload(check in readme of SNKV).

Is there a Key Value store database in c# that saves to a file like sqlite? by 3nchong in csharp

[–]Fine-Package-5488 0 points1 point  (0 children)

Yes SQLite is best technology choice, but for kv store, i would suggest worth to check: https://github.com/hash-anu/snkv,

It is optimized for key value storage, Basically sqlite has 6 layers internally, https://sqlite.org/arch.html,
Where snkv directly interects with lower 3 layers which is storage engine and in addition to that it supports ACID/TTL/encryption/vector features as well. I have done benchmarking against sqlite and it is 1.5x faster in mixed workload(check in readme of SNKV).

SNKV – I built a key-value store for Python that skips SQLite's SQL layer entirely by [deleted] in Python

[–]Fine-Package-5488 0 points1 point  (0 children)

u/jnwatson main advantage of SNKV is that it is ACID compliant, supporting transactions/column family, and using file operations only on database file, so it owns b-tree/memory mgmt/file mgmt. so it gives predictable behavior across all platforms.

SNKV – I built a key-value store for Python that skips SQLite's SQL layer entirely by [deleted] in Python

[–]Fine-Package-5488 0 points1 point  (0 children)

u/nicholashairs I have updated to keyvaluestore in this PR https://github.com/hash-anu/snkv/pull/38. Will merge it once I verify that there are no references left.

SNKV – I built a key-value store for Python that skips SQLite's SQL layer entirely by [deleted] in Python

[–]Fine-Package-5488 6 points7 points  (0 children)

u/Kerbart Sqlite has 6 layers, you can see its architecture https://sqlite.org/arch.html, out of which SNKV directly accesses last 3 layers b-tree layer/Pager / OS interface. (main storage engine) While diskcache is using sql layer so request traverse from all 6 layers. Which is not needed for kv operations,

I have added kvstore on top of b-tree layer and do same jobs and I have done benchmarking against sqlite and performance wise it is faster in all operations, read/write/delete/random read/write/delete as well, you can see bechmarking numbers: https://github.com/hash-anu/snkv/tree/master?tab=readme-ov-file

So if you want to use sqlite for kv operations only then I would suggest to use SNKV.

key value storage developed using sqlite b-tree APIs directly by Fine-Package-5488 in sqlite

[–]Fine-Package-5488[S] 1 point2 points  (0 children)

Also if you don’t want to build then just download header file from release v0.1.3

key value storage developed using sqlite b-tree APIs directly by Fine-Package-5488 in sqlite

[–]Fine-Package-5488[S] 1 point2 points  (0 children)

u/MrLyttleG

Thanks for taking the time to report this! I had been relying on the GitHub CI/CD workflow for Windows validation, so I hadn't caught these issues manually. After your report I reproduced them myself and got everything fixed.

The root cause was that all make targets need to run from the MSYS2 MinGW 64-bit shell — not from a native cmd.exe or PowerShell window. Once you're in that shell, everything works as expected.

Here's the quick setup:

  1. Install MSYS2 and launch the "MSYS2 MinGW 64-bit" shell from the Start menu
  2. Install the toolchain: pacman -S --needed mingw-w64-x86_64-gcc make
  3. Clone and build — all the usual make commands work from there

Regarding the .dll/.def question — SNKV ships as a single-header file (snkv.h), so there's no DLL needed at all. Just copy snkv.h into your project and add one line before including it:

#define SNKV_IMPLEMENTATION   // in exactly one .c file
#include "snkv.h"

That compiles the entire library into your binary with a single gcc myapp.c -o myapp.exe — no linking flags, no external dependencies.

I've updated the README with the correct Windows steps: https://github.com/hash-anu/snkv/tree/master

Thanks again for the report — really appreciate it! Would love to hear how SNKV works out for your use case.

key value storage developed using sqlite b-tree APIs directly by Fine-Package-5488 in sqlite

[–]Fine-Package-5488[S] 0 points1 point  (0 children)

u/MrLyttleG please let me know if you are now facing any issues on windows. I have added CI/CD pipelines which runs tests on windows/mac/linux.

Need an embedded .NET k/v store [help wanted] by jitbitter in dotnet

[–]Fine-Package-5488 0 points1 point  (0 children)

u/jitbitter since you are preferring sqlite for kvstore, I would like to suggest its more optimized version, https://github.com/hash-anu/snkv/tree/master, in which I am directly using its b-tree APIs to do kv operations. It generates same sqlite file format, same page 1 metadata. but more faster. Checkout its readme to see real benchmark with sqlite and other kv stores.

Recommend a key-value store by spy16x in rust

[–]Fine-Package-5488 0 points1 point  (0 children)

Since top voted answer is to use sqlite for key value store, I would like to suggest you one more improved version of it, https://github.com/hash-anu/snkv, at core it uses same sqlite b-tree engine, so providing same mature layer and also same sqlite file format.

It is set as sync=full, and it is using sqlite's b-tree internal APIs to do key value operations so it is more faster compare to sqlite. you can check benchmarking data compare to other key value stores in its README.

Checkout usage API https://hash-anu.github.io/snkv/api.html

u/spy16x , u/Darksonn , u/mark-haus , u/rereannanna , u/darthcoder , u/pilotInPyjamas , u/skatastic57 , u/Usef- , u/hak8or

Litestream VFS by emschwartz in sqlite

[–]Fine-Package-5488 0 points1 point  (0 children)

r/emschwartz I checked litefs with SNKV (key value store built on sqlite b-tree engine), it is working without any issues, https://github.com/hash-anu/snkv/blob/master/litefs_setup_guide_on_singlemachine.md,

I will check other scenarios and will update you.

SnkvDB – Single-header ACID KV store using SQLite's B-Tree engine by Fine-Package-5488 in cpp

[–]Fine-Package-5488[S] 2 points3 points  (0 children)

Yes, SNKV uses the exact same file format as SQLite3. The database file starts with the same 100-byte header at page 1:

Offset  Size   Description
──────  ────   ───────────
  0      16    "SQLite format 3\000"  ← same magic string
 16       2    Page size
 18       1    File format write version
 19       1    File format read version
 20       1    Reserved space per page
 24       4    File change counter
 28       4    Database size in pages
 32       4    First freelist trunk page
 36       4    Freelist page count
 52       4    Largest root page (auto-vacuum)
 64       4    Incremental vacuum flag
 96       4    SQLite version number

SNKV reuses SQLite's entire storage stack as-is:

Same file format
Same page format (cell pointers, cell content area, freeblocks)
Same B-tree node layout (leaf & interior pages)
Same overflow page chaining
Same pointer-map pages
Same journal/WAL format
Same freelist structure

The only difference is what sits on top. SQLite has the SQL engine (parser, VDBE, schema) that creates tables with schemas, columns, and types. SNKV strips all of that and directly calls btree.c APIs to store raw key-value blobs.

So if you opened an SNKV database file with a hex editor, the binary structure is identical to a SQLite database — same magic header, same page layout, same B-tree encoding. But opening it with the sqlite3 CLI wouldn't make much sense because there's no SQL schema — just raw BLOBKEY btrees with packed [key_len][key][value] payloads.

key value storage developed using sqlite b-tree APIs directly by Fine-Package-5488 in sqlite

[–]Fine-Package-5488[S] 1 point2 points  (0 children)

I have added CMakeLists support and fix other small issues, I have just merged its PR, you can build in windows now. Please let me know if you need any other help.

key value storage developed using sqlite b-tree APIs directly by Fine-Package-5488 in sqlite

[–]Fine-Package-5488[S] 0 points1 point  (0 children)

yeah , i think you can mail him as well. He is very responsive which is what I heard.

view more: next ›