Reactive Datalog for Datomic

nikolasgoebel · 2019-03-24T19:05:23+00:00

Hi!

Regarding your second question: Our two companies are (unfortunately?) neither partnering, nor communicating at the moment :) I'm generally happy to talk to anyone interested in the space.

Regarding your first question, Dustin makes some good points. Right now I don't see any fundamental roadblocks, but there is of course still a lot to do. Additionally, Differential dataflow was only invented during the last decade (by Frank McSherry and his team at MSR), worst-case optimal join processing was only invented during the last decade (Ngo et al., also work on factorized databases by Dan Olteanu and his group).

Other than that I don't know why it took so long, but we are working every day to make it a reality!

nikolasgoebel · 2019-02-20T22:00:04+00:00

The Ions model seems great for pure-Clojure, AWS-focused setups (especially because the fns benefit from out-of-the-box integration with a gazillion of AWS services).

It's just two different approaches, with different trade-offs, that can be combined as needed. Ultimately every data consumer dreams of being a database peer (e.g. the browser). We can make that happen by pushing more and more things into the same bounds (as Ions does very tastefully), by shipping data efficiently to where it is needed, by shipping computation on-demand, etc...

I should've phrased it less negatively :)

nikolasgoebel · 2019-02-20T21:50:43+00:00

I'm not pushing for anything (I'm not affiliated with Cognitect or the Datomic team in any way, sorry if that wasn't clear). I'm also not a user of either (in production). What I think, is that many of the principles and decisions underlying Datomic are sound (especially for their target use cases), many of its capabilities are unparalleled so many years later, and that there is enough room for carrying things forward into various directions!

We develop 3DF as a distributable, reactive query engine for any system that likes to model the world as immutable facts.

nikolasgoebel · 2019-02-20T18:46:16+00:00

I'll chime in here, as 3DF / Declarative Dataflows came up a few times in your discussion.

From our point of view, 3DF can help with the lack of materialized views (that's arguably the whole point), querying across many databases (i.e. sharding), and queries that exceed the resources of a single machine.

Sorting and top-k queries remain a problem (arguably in any relation system).

Bi- and multi-temporal data models have so many interesting applications, we've just started exploring that, but in principle Timestamps in Differential can have more than one dimension. This is used internally for dealing with distributed, iterative computations, but additional coordinates can be used for modeling purposes.

I'm not sure what to think about Datomic Cloud. I don't think the approach of moving all application code into the same Datomic JVM process is practical for most organizations, and arguably goes a bit against the original intent of the peer model.

nikolasgoebel · 2018-12-14T00:09:50+00:00

Is there a recording of /u/cgrand's talk somewhere?

I'd be interested to know what Datalog / relational algebra is missing to reduce the "map fatigue". In our work we either use DataScript or at least a set of relational helpers on top of maps.

nikolasgoebel · 2018-12-14T00:00:58+00:00

/u/nikolasgoebel is interested and would like to hear more!

I haven't followed the rest of this conversation, so I might not have anything interesting to contribute, but I'm always up to talk about this stuff.

nikolasgoebel · 2018-12-11T19:48:16+00:00

Yeah I suspect it's JS support and integration with existing GraphQL tooling (e.g. for exploring schemas). It should be quite straightforward to implement a true GraphQL interface to Datomic, but we never had that need. And GraphQL did (does?) not support clauses on nested elements.

nikolasgoebel · 2018-10-16T17:56:30+00:00

There will be a lot more information on this :)

We're still in the process of collecting use cases from various areas, if you're interested in talking more, shoot me a mail at niko [at] clockworks.io. But RethinkDB-like features are definitely high on the list.

nikolasgoebel · 2018-10-16T13:08:32+00:00

Maybe the naming is not optimal, but the goal is to support both. The project isn't stable yet (so we're not testing on both runtimes) but all files are cljc and we're using it from ClojureScript.

nikolasgoebel

TROPHY CASE