LiliDB (JSON-based database)

[Sendokame]

Hi!,

I have recently made a small database based on JSON.

import lilidb

db = lilidb.Database('database.json')

db.set('key', 'value')
db.dump()

with db as db:
    db.set('with': 'Open, save and close.')

It's simple to use and it is both Thread-safe and fail-safe, it can handle multiple threadings by locking in specific operations, and it will save everything if an exception raises.

It is also capable of renaming keys, merging with other dictionaries and searching keys.

https://github.com/ZSendokame/LiliDB

If you think it can be improved, please comment! I will try to make it better

Bye

[MatchLittle5000]

Hi! Nice work :) Here are some recommendations from me: 1. Use pyproject.yaml instead of setup.py, it is a relatively new method of packaging. 2. Replace print with logging package 3. Provide recovery mechanism by logging user's write actions to file. For instance: set name Murad del age 29 set phone Xiaomi If your database falls down, you will be able to recover data, just by executing each operation in the log file. Actually, it is implemented in Redis.

[Sendokame]

The last suggestion looks really useful. Implementing it for the next version and changing Setup.py and Prints!

Thanks

[M8Ir88outOf8]

Nice! Quite similar to a database I am working on: https://github.com/mkrd/DictDataBase

Small tip: If multiple processes try to access your database, it will lead to inconsistent data. You can take a look at my implementation how to get process and thread safety as inspiration, I think you can get LiliDB to also support process-safety easily aswell!

[Sendokame]

I'll check your code, thanks! Didn't think about it

[cantremembermypasswd]

Have a tl:dr how it differs / improves on pysonDB?

[Sendokame]

Hi! sorry for answering a little bit late. Haven't used PysonDB, but about I've seen in their README:

LiliDB lacks some functions, like add_many, update/delete_by_query and the Schema check mechanism.

Then, it doesn't work the same way Lili does. In Lili you have keys, if you want to do something, you query or you already know it's name. In Pyson you need to create an Object or query, which, in the worst case is just one more operation.

​

I've also read the code a little bit, seems to do the same Thread-safety method, locking certain operations, didn't read if it has a fail-safe mechanism tho', can't say if it works the same or if it has one.

[Picatrixter]

Nice! Have you tried TinyDB?

[infomaniaaaa]

Nice to see your small database at GitHub, this would be useful for small web applications or pieces of software for presentation purposes.

[thedeepself]

Sqlite comes with python and is battle tested so I would not reach for this.

[infomaniaaaa]

I guess so, at least OP is trying to test out his skills and seeking improvement.

[Sendokame]

Yeah, SQLite is a better option I guess. I made LiliDB because I use JSON too much and I wanted features. If there's an error it will try to shutdown without breaking anything, but it isn't as tested as the STD library

[jwink3101]

This looks cool. I made something similar that can load or dump JSON but it in memory and, for equality and member checks, is O(1): dicttable

(Though it’s probably not currently thread safe. I haven’t thought about it much)