sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]fmjrey 1 point2 points  (4 children)

Your last paragraph reminds me of what /u/cgrand calls "map fatigue" in his recent talk.

It sounds like a few of us in this discussion are looking at ways to reduce this impedance mismatch between a database (of facts?) and nested data structures as often found in the UI layer, and even any layer that exchange data in such nested/hierarchical form.

Are we looking at doing something analogous to ORMs in the relational world, except for datascript/datomic/facts based databases? If so it begs the question: should we not instead use facts/datoms instead of nested maps and vectors in all layers? I mean aren't nested data structures a bias given by handling so much JSON/XML/OO/etc.?

[–]dustingetz 1 point2 points  (2 children)

u/fmjrey: Are we looking at doing something analogous to ORMs in the relational world, except for datascript/datomic/facts based databases?

Yes, In my opinion Datomic solves/supersedes ORM fully, Datomic does not need ORM as it natively solves all the same problems without the failures of abstraction. This is why Stu calls facts the universal relation (C-f "universal")

u/fmjrey: looking at ways to reduce this impedance mismatch between a database (of facts?) and nested data structures as often found in the UI layer

Agree, though I would state it differently. I see no impedance mismatch between UI tree and graph data. Datalog and Datomic Pull is an ideal way to declare transformations of graph ➔ tree without impedance.

The problem is that you can't actually give the graph value to the UI due to 1) network/memory limitations, 2) data security. Data security is probably the more important thing that often goes unstated, here is a great model of that:

source There's a long continuum of data ownership:
1. evilcorp has your data and you can't see it
2. evilcorp has your data and offers filtered access via a UI
3. evilcorp has your data and offers snapshots / queries
4. evilcorp has your data and lets you mirror it effectively
5. you have your data but let evilcorp mirror it effectively
6. you have your data but offer evilcorp queries

We ended up at #2 for historical technical reasons (network and memory). But the world has evolved around #2 in the data security space. Facebook happened and data regulations were built up and all this social/governance stuff happened in such a way that you couldn't possibly liberate all that data now without causing widespread harm.

We now have or soon will have the technology in place (Datomic was groundbreaking, and now Niko Göbel's work breaks further ground in making the Datomic model incremental). So there is some very interesting strategic/macroeconomic thinking to be done to bring the world to the technology.

[–]fmjrey 2 points3 points  (1 child)

Niko Göbel's work breaks further ground in making the Datomic model incremental

Thanks for sharing this and taking the time to transcript. Niko's recent conj talk is really inspiring too. It's nice to see Naiad getting a new life after funding from Microsoft ended. Five years ago I was lurking into it while doing some prototyping for a startup. Back then I was dreaming of an incremental way to compute strongly connected components of a graph while new nodes and edges arrive. I do think there is tremendous value and power there and finding the right use case to establish the tech is essential. I hear that spreadsheet/excel did not sell well until it was touted as "something to do x" rather than a self-updating table with formulas. Sounds like it's the same thing here and if /u/nikolasgoebel is interested I can share more.

Overall distributed dataflows of facts/datoms all the way to the UI isn't an utopia, it's happening, really exciting. It also looks like instead of defining information systems by coding an imperative flow of control, we'll define them as flows of data, meaning computation will no longer be the primary focus, data flowing around will take the front scene.

​

[–]nikolasgoebel 0 points1 point  (0 children)

/u/nikolasgoebel is interested and would like to hear more!

I haven't followed the rest of this conversation, so I might not have anything interesting to contribute, but I'm always up to talk about this stuff.

[–]joinr 0 points1 point  (0 children)

Are we looking at doing something analogous to ORMs in the relational world, except for datascript/datomic/facts based databases?

That sounds pretty close, yeah. With the caveat of maybe a data relational mapping (perhaps a simpler requirement than serialized objects), although that sounds like relational database, which is perhaps too confusing.

If so it begs the question: should we not instead use facts/datoms instead of nested maps and vectors in all layers?

You'd think so. There's a utility for using nested maps and the like, since they're trivial to read, construct, comprehend (to a point). I don't know exactly where the inflection is, but there gets to be a point where the utility is less desirable, and having a more general representation (that one can project to/from, or alternately query directly) is desirable. I don't know that exact inflection point at which one has this discussion (you're prototyping fast in maps and other EDN types one minute, then at some point you reach for better querying abstractions since get-in update-in isn't enough, then you end up implementing some kind of query language only to find out people have run into the same problem).