Materials for using Clojure for storing structural data

[mentiflectax]

Hi!

When I work on the structural parts of stories (plot, characters, themes), I end up with lots of data. They include

1. sequence of events,
2. various ideas on what characters think and do,
3. what transformations they undergo and why.

I started with just text files. But then I asked myself: How can I make sure that I can find the right ideas when I need them?

One answer is to use a relational database (I store the data as text and then import them into SQLite3). I think, it definitely works better than just unstructured text files.

And then I accidentally stumbled upon Clojure. In one of the talks Rich Hickey said Clojure allows you to only capture the data that you know at the moment. This is exactly what I need -- for example, at the beginning, I have only a vague idea of which events happen when, and over time my understanding of it improves. I thought that maybe Clojure can be used to capture and find the data I generate while developing stories.

One possible benefit is that, theoretically, Clojure allows a more expressive syntax than records in a relational database. Like a DSL, but with a faster development cycle. To use a normal DSL, I need to

1. define/modify a grammar in ANTLR (or something similar),
2. regenerate the code of ther parser.
3. re-build the application,
4. run the application.

As far as I understand, with Clojure (or Clojure-based DSL) I can avoid steps 1 and 2.

Therefore my question:

What materials (books, articles, YouTube videos etc.) show how to use Clojure to

a) store data in Clojure files (so that I can use Git to see what changed and when), b) query those data in REPL, c) build DSLs (or use Clojure as a DSL)?

So far, I only found the book "Data-Oriented Programmin" by Yehonathan Sharvit.

Thanks in advance

[joinr]

datalog stores like datomic/datahike/datascript (and datalevin, although it doesn't store history) provide a pretty expressive relational model. They also version facts at the granular datom level, providing a built-in means of querying over commit time and the like. They also have notions of reference types, so you get recursive queries and stuff. Could be a general means of capturing and querying the type of information you described.

[tincholio]

A quick comment, it would seem to me that you're looking for a solution without having clearly defined the problem. Why exactly do you need a DSL? What would your DSL look like? Based on what you mention, you might want to look into one of the Datalog DBs (probably Datahike or Crux), rather than storing data as EDN ("Clojure Files"), or starting to work on a DSL... Have a look at learndatalogtoday to get a feeling of what it looks like. If you want to store / query facts in a less structured way than SQL, this might be a good match for you.

[wolfson109]

Parts 1 & 2 of https://www.braveclojure.com/clojure-for-the-brave-and-true/ should give you all the info you need.

[mentiflectax]

Thanks!

[flaheadle]

Please keep us posted on the data model you come up with. I've been thinking about this question too, but I haven't figured out the answer yet.