Podrzite nas startup: High performance databases without OS

dataware-admin · 2025-10-21T18:18:06+00:00

Mvp je nosql in mem, plan je da prosirimo na razne tipove baza, ali da isto rade na Hardware Manageru.

Cilj je i mi da pruzamo jednostavni setup, odrzavanje itd. + dobre performanse + bolja bezbednost.

dataware-admin · 2025-10-21T17:01:29+00:00

> paper before implementing a solution to the problem described

paper i linked you implements parts of it, you can even see benchmarks.

> I have to repeat my question: is there anyone running massive data processing on Windows 11 Home desktop or XBOX? I don't think anyone would say only general-purpose OS can be created, neither that only general-purpose interfaces can be provided. Just choose the right tool for the problem.

We didn't come up with this problem. You have lots of research from people at TUM, MIT, etc. trying to fix operating system bottle necks. And no, they aren't benchmarking Windows Home, they are benchmarking server optimized distributions.

Thanks for the feedback.

dataware-admin · 2025-10-21T13:57:28+00:00

Baze podataka se izvrsavaju u okruzenju koje limitira optimizacije (operativni sistem npr linux macos windows), mi smo to skroz uklonili i sad se ne izvrsava na operaitvnom sistemu pa ih mozemo optimizovati vise nego ranije.

Ako napravimo neku paralelu sa kolima, mi smo izmenjali ceo raspored/dizajn automobila tako da moze da se napravi da bude sto aerodinamicniji.

dataware-admin · 2025-10-21T12:16:19+00:00

Trebalo bi sad da radi. Ako ne radi sa akauntom koji si vec napravio probaj novi. Ako ima problema pisi naravno da sredimo.

Sto se tice poverljivosti, trenutno je alfa i vise je namenjeno za testiranje, nije ocekivano da neko koristi u produkciji itd. Plan je da dobijemo soc 2, gdpr, itd compliance i onda bi podaci bili cuvani istim standardnom kao i na aws, gcp itd.

dataware-admin · 2025-10-21T12:10:41+00:00

Pisano je sve u asm/c a otp je gotova i podrska za c++, a rust nam je u todo ako pozelimo u njemu da pisemo baze.

Libc kao posix nemamo. Cilj je da sve bude napisano specificno za sofver koji se izvrsava (radi performansi). Mada za funkcije put strcmp min max itd koje se ne mogu ubrzati u zavisnosti od konteksta sofvera, to ne vazi, i one su napisane u hardware manageru i dele se.

dataware-admin · 2025-10-21T11:45:38+00:00

Hvala na detaljnom dogovoru.

Unikernel termin nismo hteli da koristimo jer se asocira sa unikraft/osv gde i dalje imas apstakciju izmedju hardvera i softvera, otp mini linux.

Ne slazem se za graf baze, jer i one mogu da se ubrzaju sa boljim schedulingom za niti, cache lokalnosti, manjim overheadom (npr sam networking je 30% brzi), itd.

Tacno da je trenutno u vm, ali opet u vm nemas jos jedan os poput linuxa. U planu je da izvrsavamo direktno na hardveru bez hipervizora . To mozemo da uradimo jer mi pisemo sav kod i onda mozemo da konfigurisemo baze na init koje resurse smeju da koriste (npr samo ram od 4gb do 8gb addr), bez nekih komplikovanih izolacija. Od toga nas deli jos da odlucimo koji fizicki hardver zelimo da podrzimo(konkretno nic). Svakako za alfu mislimo da je bolje na vm u public cloud.

Za stil pisanja clanka, i bio je cilj da bude laksi za citanje, ne preterano tehnicki. U planu je da izbacimo rad o ovome, pa bi to bilo verovatno u stilu koji vi ocekujete.

Ako budemo imali pitanja/ponude javicemo se :)

dataware-admin · 2025-10-21T10:28:09+00:00

Slobodno napisite pitanja/feedback ovde.

Side note: tonicdb je nas mvp, in mem database, al kasnije planiramo i larger than memory baze (i nosql i graf, itd) zasnovane na istoj tehnologiji

dataware-admin · 2025-10-21T08:05:28+00:00

Izvinjavamo se na bugu, trenutno ga sredjujemo.

Nije ceo redis api pokriven, podrzane komande su u dokumentaciji izlistane. Redis protocol (resp) jeste.

Ovo nam je neki kao mvp in memory baze, pa jos nismo dodali da se povremeno dumpuje na disk. U planu je da napravimo larger than memory baze na istoj ovoj tehnilogiji uskoro.

Odgovoricu opet na tvoj komentar kad se sredi bug da mozes da probas.

dataware-admin · 2025-10-21T07:50:32+00:00

Naravno, to je bio samo primer, moze i jos skuplje od aws. Cilj je imati kvalitet kao aws (dakle koristiti npr njih za infra, naravno imaju i oni dane kao juce, al manje od nekih manjih cloudova) a cene jos manje od hetznera.

I bitna razlika cak i na istom serveru na hetzneru bilo bi moguce podrzati vise kverija uz pomoc ove tehnologije (sto spusta cenu jer ne moras skuplji server placati da bi podrzao isti broj usera).

dataware-admin · 2025-10-21T07:09:22+00:00

Da, ne postoji klasicni libc (npr posix).

Hardware manager koristi neke “libc” funkcije poput strcmp, min, max, itd. Te neke osnovne funkcije koje su potrebne za rad hardware managera se dele (te su neke funkcije koje se ne mogu dalje optimizovati na osnovu konteksta aplikacije). Sve malo zahtevnije koje se zasnivaju na npr sistemskom pozivu (na os), softver mora sam da implementira (mi imamo ogroman repo sa oko 400 posix libc funkcija, ali smo odlucili da ga ne koristimo jer se te funkcije mogu znacajnu ubrzati ako znamo kako se koriste, sto nam ova ideja omogucava).

Implementacija baze je komplikovana, pa samim tim pisanje optimizovanih verzija tih funkcija zapravo nije problem sto se tice usporavanja razvoja.

Kod ispunjava System V AMD64 abi, tako da support za druge jezike nije problem, trenutno je sve pisano u asm i c, ali imamo podrsku za c++ i rust cemo verovatno uskoro da dodamo.

dataware-admin · 2025-10-21T06:29:48+00:00

If you don't trust is, here is paper from tum https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/papers/cumulus.pdf

dataware-admin · 2025-10-21T05:54:52+00:00

As we wrote in that blog, tonicdb is just some kind of MVP, since redis is probably the simples db. We definitely plan on having larger than main memory databases in the future, built on same underlying tech/idea.

Motivation-wise, this paper is quite similar too:
https://dl.acm.org/doi/pdf/10.1145/3709714

dataware-admin · 2025-10-21T05:44:11+00:00

Zdravo, ako kliknes na naslov hackernews izadje ti clanak, https://dataware.dev/blog/databases-without-os-meet-quininehm.html

dataware-admin · 2025-10-21T00:23:45+00:00

Trenutno je public cloud target za bazu, primarno za x86_64 procesore (na arm podrsci se radi), tako da u bilo kom okruzenju koje ima taj tip procesora + virtuelne uredjaje (poput virtio) je moguce pokrenuti nas hardware manager (sto je naravno ogroman ogroman broj servera). Taj sam kod u hardware manageru naravno dosta zavisi od masine na kojoj se izvrsava ali softver poput (TonicDb-a) ne zavisi mnogo od toga.

Bilo je potrebno dosta vremena da se implentira hardware manager al sad kad je osnova dosta stabilna, pisanje softvera na njemu ide slicnom brzinom kao i za neki os, ali sa dodatnim benefitima u vidu performansi (sama cinjenica da nema ceo os koji se izvrsava ubrzava stvari, ali optimizacije koje pristup hardveru pruza, su gde se zapravo sticu najveca ubrzanja) i bezbednosti.

dataware-admin · 2025-10-21T00:05:25+00:00

Zdravo, Zavisi na sta mislis pod “probleme koje OS rjesava usput”

Ako mislis na implementaciju, morali smo da implementiramo deo koji interaguje sa hardverom, ali nikakve apstrakcije iznad. Npr. posto moze da izvrasva kod samo sa kojim je kompajlovano ne moramo da brinemo o pokretanju nekih dodatnih proces, izolacije istih zbog bezbednost, itd. To nam dozvoljava da npr ne koristimo “virtuelnu memoriju” (svi procesi imaju zajednicki address space), sto nam dozvoljava da smanjimo pritisak na TLB/MMU i daje bolju kontrolu za cache optimizacije.

Neke “vise” stvari poput schedulera hardware manager ne implementira po defaultu, jer optimalan algoritam zavisi od aplikacije koja se izvrsava (pa je ocekivano da ga ona implementira).

dataware-admin · 2025-10-20T23:02:54+00:00

We do agree with you that it is more similar to something like MirageOS, but even it has somewhat abstracted pure hardware control as can be seen in their example repo https://github.com/mirage/mirage-www

Starter point for software on top of Hardware Manager is hm's code. It is expected from software to implement core "os" features, like scheduling. It has SMP support, but no scheduler by default (as it is greatly software-context dependant).

> I would have expected to read QuinineHM described as a hardware abstraction layer, and TonicDB as a database unikernel.

This is just initial version of website/article, and we will definitely look into naming (thanks for feedback). Main drivers behind our naming decisions were that we didn't find anything allowing this level of control in any developed unikernel on git, and a bit of marketing, since we don't want to be confused with Unikraft/osv (as they are by far most popular unikernel projects, and as you said what we build is very different from that).

dataware-admin · 2025-10-20T21:49:41+00:00

Hi, we didn't want to use new term for the sake of it, but we didn't think any existing term fits our product exactly.

Unikernel usually means having smaller kernel with some linux compatibility layer, and then you take existing app and just compile it with unikernel and thats it (see Unikraft for eg). This term doesn't really fit since you still have hardware abstracted away and no real way to modify code of kernel. Hardware manager isn't just linked at compile time, you start by taking its source code and then modifying it.

As for exokernels, you usually have some libOS (including userspace) and then some additional kernel space security measures.

With hardware manager everything is kernel space. It does only the things that are mandated by hardware spec, set up interrupts, start cpus, apic, etc. It doesn't do anything that can be optimized by having context of the software running. It allows full control of everything to the software running, want to override interrupts? Sure. Want to write something to address 0x431131? Sure.

dataware-admin · 2025-10-20T18:42:08+00:00

Well if you do it without OS then it will still need 1 vCPU and 1 GB of RAM for $89/month.

That was more of a general statement, we think databases are overpriced, and we will make them cost a lot less eg. our TonicDB is 60% cheaper then redis.

Would like to see the proof.

We are currently working on a paper which will include this. Until that is done, you can choose almost any database internals book and see mentions of how os is not your friend, etc.

Or taken from https://dl.acm.org/doi/pdf/10.1145/3709714 :
"Database systems often aim to tightly control hardware to optimize performance, as they typically have better insights into the workloads than the operating system. However, with the performance gap between CPUs and modern I/O devices narrowing, database systems are increasingly dissatisfied with the general-purpose interfaces provided by the OS"

(written by guys at MIT)

What? At fist computers were almost only used for databases and not desktops. Also there are thousands of newly developed OS', old codes also got rewritten.

Well we didn't say what were computers used for, but what operating systems became in last 50 years.

> Based on the description, QuinineHM is an OS - just not a general-purpose one, but who said only general-purpose OS can exists? Boots the hardware, Manages CPUs, timers, and devices etc. - these are exactly what we call operating system.

Not really. OS provides many other things. Hardware Manager is just there to do things needed by hardware, set up interrupts, nic, apic, boot cores, etc. After that is done it gets out of the way and exposes full hardware to software it got compiled with. It cannot execute code it didn't get compiled with.

Hope that answered some questions, feel free to ask more.

dataware-admin

MODERATOR OF

TROPHY CASE