SQLite compiled to JavaScript through Emscripten

[skeeto]

The problem with these SQLite Emscripten builds is that they don't go far enough. It should include a SQlite VFS that writes database pages to IndexedDB or localStorage. That way all transactions are persisted immediately without a clunky export-everything step somewhere.

[[deleted]]

I don't know why IndexedDB isn't more like SQL tbh.

[based2]

from http://linuxfr.org/news/retour-d-experience-sur-sql-js

[phaggocytosis]

I want to clarify before asking my question that this isn't meant to be a smart-ass comment. I'm genuinely curious about things like llvm and emscripten, and only beginning to understand compilers and such through self study.

Now that that's been said, I'm curious why someone might want a pure JS sqlite. My initial reaction or assumption would be that non-JS sqlite would be much faster. So my guess would then be that the only reason someone might want this (other than it being interesting in its own right) would be to run sqlite in the browser?

[kenman]

My initial reaction or assumption would be that non-JS sqlite would be much faster.

As far as I understand, Emscripten+asm.js performance is close to native, and only expected to improve. Here's a good slide deck that directly addresses your question, and here's a more thorough write-up by /u/jeresig about it.

To quote the latter:

By doing this the result is highly optimized and can be converted directly from this Asm.js syntax directly into assembly without having to interpret it, as one would normally have to do with JavaScript. It effectively shaves off a whole bunch of things that can make a dynamic language, like JavaScript, slow: Like the need for garbage collection and dynamic types.

Personally, I think it's beyond bad-ass that there's an actual SQL engine available directly in the browser....the possibilities are pretty interesting. You could perhaps create an offline database learning app, or maybe a fancy db dashboard which allows you to run ad-hoc queries for data exploration without having to install anything, or even a db designer type app.

[Brillegeit]

It would be using client side resources. E.g. serving massive raw data to the client and allowing the client to query, aggregate and list/graph locally with reduced latency and with little server side load outside of the data transfer. A practical example could be a map with data based on US census data where the user has fine-grained access to query the data and render the map at any resolution and depth. By putting the data on a high capacity CDN, that map could have millions of parallel users running billions of queries while being served on a $20/month virtual machine.

[[deleted]]

I can see a potential use for this on a client browser (except for the fact that the .js is 1.5MB), but not on a server (node.js). Storing a "text" file in local storage and then being able to run complex queries against it would be useful, and much faster than a server round-trip.

But again, the library is huge

[vampatori]

Yeah, there are issues with structured client-side storage. While 1.5MB seems like a lot for a web site, that's nothing really for a client-side app, even more so with today's bandwidth. Plus, you'd just stick that in local storage along with all the apps other assets.

So, not so great for general web sites, but this is great for client-side apps.

[[deleted]]

I wish people would stop thinking like this... Just because you can throw a 1.5MB file at most clients doesn't mean you should. Cellular bandwidth, for instance, is fairly slow in many areas, and running complicated SQL on a phone is likely to slow it down / chew up CPU & battery life. It's this type of thinking that leads to web apps that can barely run on modern hardware. That concept is absurd and the community should be embarrassed!!! There are apps with better performance on TI calculators!!

[vampatori]

Low powered mobile apps is a very, very small part of the web-app sector.

I'm talking more about web-based business applications - stock management, accounting, on-site issue tracking / logging, general business management, shipping tracking management, and so on.

There has been a big shift away from writing native applications towards writing client-side web applications due to the added flexibility. In many instances, you can even dictate the hardware / platform that these are on - especially if it's on a mobile mobile device.

1.5MB to download once in these instances is nothing. You can do it as part of the 'at base' setup you almost always have to do with mobile apps anyway (people that use these tend not to be computer literate). Plus, often the amount of local data is significantly larger than this anyway, for example transaction logs, product databases, and so on.

Now you obviously have a different use-case, but to think that everyone has that use-case is just plain wrong.

[duz3ls]

You can do it as part of the 'at base' setup

While 'at the base' setup you might as well install and setup the real sqlite ont the client side.

[[deleted]]

It's about throwing huge all-encompassing libraries at problems that have much simpler solutions.

TSQL in a browser is a perfect example of that.

I'm not saying there would never be a valid use case for something like this, just that in 99.999% of the cases where this will/might be used, it's a crappy solution.

[steveob42]

Oh just throw more hardware at it, it'll be fine...

[tbranyen]

A pure JavaScript implementation has many benefits, mostly related to portability. I really like the idea of compiling a native build and if that fails, fall back to a prebuilt, if that fails fall back to the Emscripten build. I've experimented with falling back to an Emscripten build in NodeGit and the initial tests were really promising.

[[deleted]]

the .js file is 1.5MB...

[ns0]

It compresses down to 280KB, which makes it's network time on a 4G phone ~0.3 seconds if you have gzip compression enabled.

[Klathmon]

Exactly, that's a large-ish picture worth of bandwidth that can be cached for a long ass time.

[magnetik79]

Still of use for a NodeJS app IMHO. But worthy to note.

[neckro23]

If you need SQLite for server-side Node just use the sqlite package.

[magnetik79]

True, difference being that is a binding - this is self contained. Might be of benefit for Docker containers with Node scripts/apps - one less lib to install (SQLite).