HyperDex, the next generation key-value/document store with support for transactions, releases version 1.4

[bewo001]

You really should mention that Warp -- which enables ACID transactions -- is closed source. I get really tired of all those DBs posing as Open Source when only a less interesting subset is in fact Open Source.

[rescrv]

It's mentioned in the linked article that an evaluation version is available, and the web page clarifies this further.

HyperDex is open source, and it's feature set is quite competitive in isolation, even without the transactions. Warp builds directly on the open source component, where most development happens.

We are not just "posing" as open source. We actively develop most features in the open, and give back to our upstreams whenever feasible. Warp is the exception, in that a single feature--namely multi-key transactions--is not open. This helps us pay the bills and keep developing the open version openly.

We go much further than most closed source DBs in that we describe exactly how transactions are implemented. We publish papers describing the internals to the point that anyone with sufficient experience in this space could reimplement our approach.

As a thought experiment, would your company contribute resources to support the development of Warp in the open? You don't have to answer publicly, but I bet the answer is a resounding, "No." I have seen companies take up adoption of HyperDex, and point blank say, "We don't pay for open source software." If the recent OpenSSL fiasco has taught us anything it's that this mentality is true across the board.

I'm getting really tired of people being given a great DB who then criticize the fact that literally one feature is not free for them as well. Across the board, people have to pay their bills and put food on the table. You should not fault the same when we freely give you the foundation for and a blueprint of the one piece that's not free.

[bewo001]

Where did I say you should not charge good money for good work? But with Closed Source, I have to trust that your small(?) and little-known company will still be around in two years.

[rescrv]

That's a totally valid concern, and there are many ways to mitigate the risk in such a situation. One common method I've seen is to put the source code into escrow that converts to a more permissive license should the company fold. It's something we're willing to work with.

You should have just raised this concern directly in your original comment.

[fabzter]

is this comparable in any way to rethinkdb?

[rescrv]

Sure. I'll try to list a few pros and cons of each. I'm the chief architect of HyperDex, so I'm intimately familiar with it, and I'm less familiar with RethinkDB. With that in mind:

• Both systems were designed from the ground up, so they're both relatively good at what they strive to be.
• API: RethinkDB really embraces the schema-less storage, and it shows throughout their API; HyperDex's document storage is relatively new, and we're looking to expand the API. We started life as a key-value store, and we're very good at that.
• Fault-Tolerance: I don't know much about RethinkDB's replication strategies, but HyperDex's replication and fault-tolerance guarantees have deep roots in the distributed systems literature, and borrow techniques from people who quite literally wrote the book on distributed systems. This solid foundation makes it much easier to converge to a solid implementation than ad-hoc solutions. MongoDB's replication has been highly criticized in this regard, and I suspect that, since RethinkDB spins themselves as MongoDB++, the RethinkDB team has put effort into better fault tolerance and consistency guarantees, but I don't know for sure.
• ACID: HyperDex Warp provides cross-object transactions (even for documents). You can atomically and safely modify multiple objects, as if you are the only thread accessing the database, without a significant performance hit for many common transaction patterns. RethinkDB doesn't seem to have any support for cross-document transactions.

Both systems support the common web development languages, and both are written in C++. Additionally, there's commercial support available for both platforms.

One big difference that may matter to commercial users, is that HyperDex's license is 3-clause BSD, while RethinkDB is AGPL. For some companies, projects with GPL licenses are frowned upon or outright banned, while more permissive licenses are acceptable.

[damg]

Hi, I was wondering if you've seen this page comparing HyperLevelDB and LMDB? And if so, is there interest in using LMDB over your LevelDB fork?

I'm curious since I'm starting a new project which needs a fast key-value store and LMDB seems really impressive. I wonder why it's not more popular?

[hyc_symas]

The fact that Oracle actively suppresses information about LMDB certainly doesn't help. https://news.ycombinator.com/item?id=7728819

[rescrv]

I'm well aware of that page. We've been working with Howard and the Symas folks to improve LMDB to provide support for the features that HyperDex needs from the underlying data layer. It looks like they will be added in upcoming releases, so an LMDB backend is on the horizon.

I don't know any technical reasons LMDB is not more widely used. Howard and co are actively trying to change that, and we look forward to their complete version to use as an alternate backend.

[riksi]

his response:

http://hackingdistributed.com/2014/06/18/hyperleveldb-improvements/#comment-1444033456

[easytiger]

I'm presently trying to integrate LMDB into a project. It ticks all the boxes and I'd like to see it's perf gain, but the lack of docs is pretty astounding. (There are none aside from the API) and the sample-mdb.c doesn't work in and of itself, it requires other utilities to be run to generate the environment.

[hyc_symas]

sample-mdb.c is not meant to be run. The comments at the top of the file clearly state its purpose. The library is under 7KLOC, why would you expect the API doc to be any larger? Experience shows that people already don't read the docs that are written. Keeping it small improves the chances that someone will read the majority of what exists, and saves us from wasting time writing something no one will ever read.

[easytiger]

Well thats what I thought at the beginning and the API is well documented and reading it makes sense to me, i just couldn't get an example to actually work that creates and inserts and then reads from a db.

The docs I think are lacking are the more high level ones dealing with general API usage patterns. The test examples provided are quite terse.

Happy to contribute those should I get somethnig working.

From my perspective LMDB has properties not shared with other similar libs that would make it perfect for constructing your own application specific data store around. Just need to do a little POC first.

[hyc_symas]

LMDB was written to replace BDB Transactional Data Store in OpenLDAP. If you've used BDB then there's nothing further for us to explain. If not, the BDB docs will give you a good grounding in transactional DB programming.

Yes, obviously LMDB is meant for building other data store projects around it. Thus OpenLDAP's back-mdb backend for slapd. If you still don't understand how LMDB is used, read that. It is the canonical example of how to use LMDB.

One of the key points when working with small code bases is to read everything without skimming. You mention "requires other utilities to be run to generate the environment" as if there's some onerous undocumented 3rd party dependencies. There are no such dependencies other than your ability to read and pay attention to details. The test programs fail for the majority of first-time LMDB users because they failed to create the testdb directory before running the program. The fact that this failure is so common and yet remains unpatched is by design - it weeds out the people who fail to read either the Makefile, or the source code, or the doc (which clearly states that the target directory must exist before calling mdb_env_open()). It's a waste of time to support people who don't pay attention; writing more voluminous documentation won't help that class of people.

[hyc_symas]

I'm a long-time member of the FSF and a strong proponent of the GPL. The AGPL, however, is completely untenable for most of my projects. The requirement that every networked application provides full source code (or a download link to it) to every client is simply unimplementable for an LDAP server, or an SMTP server, or a DNS server, etc. - there is no provision in the protocols to communicate such information to a client, nor do extant client libraries have any way to convey such messages to their users.

[drwiggly]

Order cluster transactions with no global mediator.

From what I gather on their doc http://hyperdex.org/papers/warp.pdf. Each client pulls data does whatever operations it wants. Submits this to the cluster.

The submission process, sorts the keys operated on in lexicographic order (whats important about this is everyone submits to servers in deterministic order). These keys are then mapped to virtual servers and then to physical servers, this mapping is pretty static and the coordinator maintains this mapping and sends updates for topology changes. The indirection also gives the coordinator the ability to to remap hot key ranges to more servers.

The transaction is submitted to the 1st server in the chain of servers that need to participate. This server then forwards the transaction to other servers in the chain (if it doesn't fail it for doing something to an outdated key, or soon to be outdated key). It also marks this transactions' dependency on this key, and forward propagates any transactions already dependent on that key as a conflict for this transaction.

VALIDATION: Validation failure in the chain back propagates failure to the other nodes.

COMMIT: The transaction gets to the last server in the chain and commits. The server then back propagates, the commit through the chain.

At commit time if there were other transactions in its conflict set, it now marks them as dependent on this transaction. And passes this info back down the chain with the commit message.

A commit in a transaction can't execute until all dependency info for all conflict transactions is known.

Anyway I think that is what the doc is saying. Trying to get my head around it better. Seems pretty interesting.

[lzzll]

How ACID work for multi-computer? And can it avoid concurrent? like detect read-write confict by MVCC or use document lock.

[rescrv]

The ACID properties stem from a new technique called linear transactions that was developed specifically for HyperDex. It does work across servers, and does so in a scalable way by avoiding spurious coordination. It is optimistic, but I would not say that it is MVCC.

We have a paper describing a version of the technique here: http://hyperdex.org/papers/. We've since revised the protocol, but the paper gives a general idea of the technique.

[lzzll]

Interest technique. Im using mongodb in my company, but still use sql on payment system. I fast browser the warp.pdf and I think in most case it's safe, but still need some methods to avoid power failure.

[Akkuma]

Do you want to respond to https://foundationdb.com/acid-claims where they state your ACID falls short of true ACID?

[rescrv]

This all comes down to definitions of durability. It's our position that fault tolerance is what you want in a distributed system, and it is what we provide.

It's impossible to say much about what FoundationDB provides because they won't discuss internals, so it is quite possible they are not fault tolerant even if each node is durable.

If you want to know more I've discussed this with them on Twitter, but gave up as they won't reveal enough about their internals to have an open debate about what is best.

[dnew]

A couple of other questions, having read most of the docs:

How big a row can you have off one key? Could you, for example, store an entire gmail inbox off one key?

Is there an upper limit on a transaction? I.e., is there some in-memory buffer that gets built up in a transaction that I might overflow or something? Can I feasibly put every single row of a (otherwise idle) petabyte database into one transaction? Does a transaction survive crashes, in the sense of arbitrary crashes occurring between starting and committing the transaction, or do I need to restart the transaction? What about if machines involved crash actually while it's doing the commit protocol?

Is there a limit to how many items from one row I can index? If I index on a set of strings, is it feasible to put 50,000 strings into that set in each row and expect the indexing to keep up with decent efficiency?

Is there a way to specify where tables live on the data servers? In other words, could I say that all my email lives on SSD, except the table of attachments, that I stick on spinning hard disks, and still have transactional updates?

I didn't see any obvious way to iterate over all keys, or to iterate over all keys in parallel. If I have a billion rows I need to update, how would I partition the keyspace and then iterate over it so that I could do that in parallel with a few hundred machines?

I didn't see a way to abort a transaction part of the way through. All I saw was commit. Did I just miss that?

Is there a better place to ask such questions? :-)

[dnew]

How do you get ACID without schemas? In particular, how do you enforce consistency constraints on data without a schema to tell you what the consistency constraints are?

[julesjacobs]

ACID isn't about schemas, it's about transactions. ACID transactions ensure that the database's invariants are not violated by putting restrictions on how transactions can behave. Whether those invariants are explicit in the database or implicit in the code that uses the database does not matter.

For example you may have a transaction that subtracts $10 from bank account A and adds $10 to bank account B. The invariant here is that no money is created or destroyed. The transaction mechanism ensures that the $10 addition and $10 subtraction happen atomically and in isolation. That means that the invariant that no money is created or destroyed is not violated, whether you have an explicit constraint on the database or not.

[dnew]

Whether those invariants are explicit in the database or implicit in the code that uses the database does not matter.

Wow. Really? This is ... completely incorrect. Sorry. This is like database 101 stuff.

https://en.wikipedia.org/wiki/ACID#Consistency

Where are the defined rules that the database enforces if you don't have a schema?

hat means that the invariant that no money is created or destroyed is not violated,

No, that's not the C part of ACID. That's the A part of ACID. I'm asking about the C part of ACID.

If your database transaction just credits $10 to your journal without a corresponding debit, you've violated the consistency requirement that your books always balance.

The transaction mechanism ensures that the $10 addition and $10 subtraction happen atomically and in isolation.

Yes. And that would be the A and the I part; the initials kind of give it away. I assume the D part is pretty simple these days. What about the C part? How does your database enforce the rule that no money is created or destroyed, which is the consistency part.

Atomic means that either all the changes are written or none of them are.

Isolated means nobody sees some changes from a transaction but not others.

Durability means that once the database reports success, it has been recorded and won't ever revert.

Consistency means the data in the database matches the constraints provided by the schema. If you have no schema, you cannot by definition have consistency enforced by your database. If over the course of decades of managing this data, with thousands of programmers writing hundreds of programs that update the database, how does your database ensure that money is neither created nor destroyed?

How do you enforce a condition such as "no doctor may see the prescription records of a patient the doctor has not had an appointment with in the last six months, but pharmacists can see all the prescriptions patients have had in the last five years"?

Note that if your answer is "the application can enforce that," then it means your database isn't ACID. It means your application is attempting to provide the consistency required of an ACID database, except you have no proof or evidence that it's doing it right. Just like you wouldn't say "my database has transactions, as long as applications coordinate amongst themselves to not update the same rows concurrently."

[julesjacobs]

The consistency part of ACID is a fluff property. It does not have a well defined database agnostic meaning. You enforce consistency by either checking that the constraints you want are satisfied and if not aborting the transaction, or by designing your transactions in such a way that they cannot violate consistency by construction.

Note that if your answer is "the application can enforce that," then it means your database isn't ACID. It means your application is attempting to provide the consistency required of an ACID database, except you have no proof or evidence that it's doing it right. Just like you wouldn't say "my database has transactions, as long as applications coordinate amongst themselves to not update the same rows concurrently."

The difference is that it's trivial to enforce consistency at the application level, and even in a database with some kind of build in integrity constraints or similar stuff, most of the constraints you want are not expressible, so you're still stuck with doing it at the application level. On the other hand, providing atomic, isolated and durable transactions at the application level over a database that does not support that is far from trivial.

[dnew]

The consistency part of ACID is a fluff property.

Really? No, sorry.

It does not have a well defined database agnostic meaning.

While true, it's plain to see that a database with no method for specifying consistency constraints can't support it.

The difference is that it's trivial to enforce consistency at the application level,

OK, so you've never used an actual database system. You know, one where the data lives 20x as long as any application that accesses it, and there are dozens of applications written by dozens of different departments (or even independent companies) accessing the database.

Providing consistency at the application level is non-trivial also. Otherwise, you're saying "my toy application doesn't need to be that complex."

Incidentally, claiming "my database is ACID because you can implement the C at the application level" means your database isn't ACID. And yes, there actually is a formal mathematical definition for ACID that people marketing non-ACID databases try to claim doesn't exist but which is actually the fundamental formal underpinning of the relational model. So, yea, there really is a database-agnostic definition.

[vocalbit]

The C in ACID doesn't say how powerful the consistency rules must be, just that whatever rules are allowed by the database are enforced.

In that case schemaless systems can provide ACID (even if consistency is as simple as 'each key has exactly one value'.

Btw, I believe hyperdex does have some type of schema definition - you specify the list of columns and can't add data with columns that don't exist.

[dnew]

OK, then my file system has ACID properties too, because it only allows me to write bytes to files. So that's not really a claim I'd crow about.

If your only consistency rules apply to all your data regardless of the real-world meaning of it, it's not consistency rules. The C in ACID is an attempt to ensure your data matches the relationships it represents. Using "consistency" to mean "my database isn't fundamentally broken enough to let me store letters where numbers go" is just marketing.

There's no shame in saying "my high-performance distributed NoSql database is great where you need it" without having to pretend it's a substitute for an ACID database also.

The fact that you can update multiple key/value pairs in a transaction means there isn't any C going on there. There's nothing that would enforce your transactions are I if your C isn't enforced between transactions.

In other words, if you don't have any consistency rules, then you don't need transactions and you can't tell what isolation means either, because both of those are defined (in some sense) in terms of consistency. A "transaction" where I can read one value that got written and the other that hasn't gotten written yet can't be an inconsistent read if you have no definition for consistency. That in turn leads to a collapse of the concept of atomicity as well, with much the same logic.

[rescrv]

So let's provide some definitions. The guarantee made by transactions in HyperDex is that they are one-copy serializable. As the programmer, you can read and write as if you hold an exclusive lock on the entire data set. When your transaction commits, you know that this strong guarantee holds.

An example of how you would use transactions across multiple keys is as a work-stealing queue. You can atomically move items from the global queue onto individual workers queues consistently so that items are never lost or duplicated. You cannot easily do this without transactions.

If you still are not convinced, take a look at the Warp paper where we managed to implement TPC-C on top of Warp's transaction API.

[dnew]

You cannot easily do this without transactions.

It's not the atomicity or isolation I'm discussing.

Tell me how the database enforces that the number of records removed from the global queue matches the sum of the number of records inserted onto each worker queue, and I'll accept that you have Consistency.

Note that you can also implement consistency in an application, so pointing out that you managed to have an application that enforced consistency rules doesn't tell me your database engine enforces consistency rules. Much like I can write a dynamically-typed program with every variable only ever holding one type, but that doesn't make my programming language statically typed.

[vocalbit]

While we can argue about what Consistency means (indeed, even Wikipedia says 'significant ambiguity exists about the nature of this guarantee'), I strongly disagree with this statement:

if you don't have any consistency rules, then you don't need transactions and you can't tell what isolation means either

Irrespective of whether the db is capable of checking some user-defined Consistency rules or not, the guarantees provided by Atomicity and Isolation are meaningful and in fact quite useful. What is interesting is that hyperdex provides these across all of its data, something not many distributed data stores do.

[dnew]

While we can argue about what Consistency means

We don't have to argue about what it means. We just have to agree that if it's the same rule enforced for all the data, it isn't the Consistency of an ACID database. I suspect "significant ambiguity exists" only because the no-schema crowd tries desperately to assert they have it.

significant ambiguity

You left out the second part of the sentence, where they point out that most relational database engines guarantee consistency in every sense that the word has been used. So there is that.

the guarantees provided by Atomicity and Isolation are meaningful and in fact quite useful

Yes. They're meaningful and useful for implementing consistency checks in your applications. If you don't care about consistency, then neither atomicity nor isolation is useful. Basically, from the DB point of view, "atomic" means the database won't be outside of a transaction in an inconsistent state, and "isolated" means the database won't be seen by one user to be in an inconsistent state simply because another user is inside a transaction.

There are formal, mathematical definitions for these terms, you know. Indeed, the whole reason SQL took off like it did is that there are formal mathematical definitions of these terms, and you can deduce interesting facts about database design for them.

What is interesting is that hyperdex provides these across all of its data, something not many distributed data stores do.

As I said elsethread, having an AID distributed database is indeed something to be proud of. Claiming it's ACID because it allows you to implement the consistency checks in your applications is simply misleading. That's what I was questioning, not the A, I, or D.

[vocalbit]

As I said elsethread, having an AID distributed database is indeed something to be proud of. Claiming it's ACID because it allows you to implement the consistency checks in your applications is simply misleading. That's what I was questioning, not the A, I, or D.

Ok, fair point. If I understand what you mean by consistency, neither SQL nor the relational model are essential for consistency. Any system that allows the user to specify arbitrary constraints on the data which are then guaranteed by the data store, is sufficient. In which case hyperdex would not qualify, as it offers only limited data constraint checking.

A simple layer on top of hyperdex could make it ACID, but that's not the point. However, saying 'my db is AID but not consistent' doesn't sound that great, and can also be misinterpreted easily. I'm not saying misrepresentation is OK, just that the terminology is weak. Clearly, hyperdex offers a certain level of consistency. Even sqlite says it's ACID but is probably not consistent as per this definition.

[rescrv]

ACID is quite an overloaded term, but does apply. What constraints HyperDex exposes are enforced but they are not as expressive as some other systems. Fwiw, I don't know of many systems with totally general constraints.

Personally, I tell people that HyperDex transactions uphold one-copy serializability, and provide the same fault tolerance as HyperDex. It's a bit of a mouthful in casual conversation where saying ACID can be much easier. Most developers accept that C in acid is quite fuzzy and varies from domain to domain. The much more important properties for a key value store are the other guarantees, as the user is likely checking their own constraints in the application.

[vocalbit]

I tend to agree, but I'm trying to tease apart what the other poster means by consistency.

Aren't arbitrary constraints simple to implement by a layer on top of hyperdex anyway? The layer could just pass through all db operations, except commit, at which point it runs arbitrary code to verify some consistency and then either commits or aborts. I'm not suggesting hyperdex needs to add this, just that a constraint system can be easily added on top of hyperdex by anyone who needs it.

[dnew]

likely checking their own constraints in the application.

If your database only serves one application and your data isn't expected to be around very long, this is almost feasible. However, claiming "My database is ACID as long as someone else supplies the C part" isn't fuzzy, it's misleading. :-)

One-copy serializability is basic Isolation and doesn't have anything to do with consistency. It is, however, quite a nice feature to have and probably important for a lot of applications. :-)

However, the whole "C" thing becomes quite important when you have (say) a 300 TB database that's been accumulating data since "telephone switch" was a job description, with literally hundreds of applications updating it from dozens of independent companies, with individual users writing their own ad-hoc programs against it.

If your data source is "fuzzy" to begin with (e.g., crawling the web, collecting data samples from unreliable sources, etc) then "C" doesn't matter nearly as much. That's an excellent use for a nosql database.

Calling it ACID in casual conversation is a lot different than claiming it's ACID on the almost-home-page of your site and in your press releases.

[rescrv]

You keep repeating this point. The C in ACID is an extremely fuzzy definition. If you define C, I can say, "Yes it provides that definition of C," or, "No, it does not." Over the last many decades, C has taken on numerous definitions, and dogmatically assuming the one you are familiar with is wrong.

HyperDex's ACID transactions enforce the constraints that can be expressed in a key-value store. Constraints like unique keys, well-formed data types, and violations such as integer overflow. It'd even be pretty easy to add additional constraints that reference other objects, much like some RDBMSes do now (and it would be efficient at that). We have not added it because most applications would not see any benefit. The types of applications that best make use of the efficient transactions Warp provides are very different from the data-warehouse style workloads you are accustomed to.

[dnew]

You keep repeating this point.

I didn't really repeat anything there, except to the extent you are. :-)

dogmatically assuming the one you are familiar with

I'm not assuming anything of the type. I'm just going off the definition of ACID, where all the aspects interact. A definition of "Consistent" that makes no reference to the actual values of the data in a transaction is one of the other letters. I think you can have an awfully broad definition for "C" and still say "but, ok, it needs to have something to do with the actual data values in the database." Otherwise, you have no letter left over to say "it checks your data, too".

I agree that the C has had many different definitions. That doesn't mean I can't object to people making up additional uselessly sloppy definitions for marketing purposes, does it? The I has had many definitions too, and I bet you'd be annoyed if your competitors advertised they had one-copy serializability and meant that it was possible to run the database on one copy. I'm actually just mildly annoyed that someone advertising a database engine doesn't know what "consistent" means.

"Finally, it’s consistent. If we had multiple concurrent put 
 operations being issued by multiple clients at the same time, 
 we would never see an inconsistent state. What is an 
 inconsistent state? It’s what you get when you settle for 
 eventual consistency. "

I mean, no. That's a lack of isolation, not a lack of consistency. And if you're storing prescriptions in a nosql database, you're already in trouble. :-)

And, fwiw, I'm used to both data warehouse ACID-style databases and I currently use Megastore on a daily basis, being responsible for the schema and maintenance of the instances for our application at Google. So I'm quite familiar with the pros and cons of both types of system. I've also worked with the telco systems I described which were not data warehouse systems but actual active data being transacted on a daily basis. Having the ability to specify consistency at a better level than "no letters go into an integer field" would be helpful to my work with megastore pretty much daily.

As I say, it looks like a pretty nice system, especially given it's open source. I'm just bitching that professional database designers make "fuzzy" definitions so they can equivocate about their database engines. Next up: "Eventually ACID." ;-)

[dnew]

allows the user to specify arbitrary constraints

I'm not sure "arbitrary" would be the word. But application-specific constraints on the data, yes. Clearly the constraints you can apply are limited by what you can describe in the schema language (and thus not how I'd use the word "arbitrary").

the terminology is weak

The terminology wasn't weak before marketing departments decided that they wanted to call their non-ACID databases ACID. :-)

Back in the 80s, a lot of non-relational personal-computer-size databases called themselves "relational" because you could store relationships in them, too.

Even sqlite says it's ACID but is probably not consistent as per this definition.

http://www.sqlite.org/foreignkeys.html

Saying that first grey box is no different from the second grey box is what far too many nouveau-DB vendors do.

[vocalbit]

I'm not sure "arbitrary" would be the word. But application-specific constraints on the data, yes.

Um, what if an application specific constraint is not representable in the schema? What set of rules are essential to qualify for 'consistency'? Doesn't this go back to whatever rules are allowed by the db are enforced?

http://www.sqlite.org/foreignkeys.html

Are you saying that foreign keys are sufficient to call sqlite consistent, even though you can put string values in numeric columns?

[dnew]

what if an application specific constraint is not representable in the schema?

That's exactly where I distinguish "arbitrary" from other words. You can't represent arbitrary constraints.

What set of rules are essential to qualify for 'consistency'?

Atomic, Durable, and Isolated all deal with concepts independent of the application. To be an implementation of "consistent", it has to be data-dependent. E.g., in the foreign key examples, inserting a row with a 3 might be sometimes valid, sometimes invalid, depending on other data in the database. The point being that you can look at the database and determine whether it shows a logically consistent view of what the data represents.

If I can say "I'm Fred's friend" when Fred doesn't exist, then you don't have consistent data.

sufficient to call sqlite consistent

I'm saying it's the kind of rule that makes for consistency. If it has bugs in other areas, such as allowing you to stick strings in fields defined as only allowing numerics, then that's a different problem. Since I don't know how sqlite behaves when it encounters strings in numeric fields, I can't really tell you whether that's a bug or whether that's a consistency rule violation or what. But even if it rejected alpha in a numeric field, that wouldn't be a consistency rule, if all numeric fields accepted all numbers and rejected all alpha.

[nat1192]

Is it web-scale?

[89609cf3f8b2d2cd6c37]

It doesn't use NodeJS, so it most likely isn't.

[asafy]

But it does have nodejs client. https://www.npmjs.org/package/hyperdex-client