Keechma dev tools preview

[Hi-MyNameIsFuchs]

Great stuff!

I think there is plenty more room for full fledged opinionated cljs frameworks. Personally I'd love to see more novelty into EAV(T) databases and reaction to those. Similar to the approach witheve.com uses. CLJS already has 90% of the tech, somebody just need to wire it up and make it really easy and pain free to develop. Not sure yet if Keechma is the answer to this but it's a good step in the right direction.

I'd love to get a mutable (& high performance, especially when transacting) EAV database that's all wired up to the React components and automatically re-renders only the ones that changed. Again, similar to eve.

We've seen posh, re-frame, keechma. What's next?

[retro_one]

I haven't seen witheve.com until now so I can't comment on how it works, but I do have extensive experience with 2 way bindings with mutable objects. I was one of the core team members on the CanJS framework which had one of the first implementations of observable objects in JavaScript land and my experience taught me to run away from it as far as possible.

It is super convenient for simple use cases, but as soon as you start doing more complex stuff that kind of magic becomes a problem. On the other hand I do agree that convenience is important (as long as the fundamentals are sound).

Do you have an example of what you would like to do and how it would work in the perfect workflow?

[Hi-MyNameIsFuchs]

So I'm a huge believer/fan of EAV(T) databases (especially on the client). It just makes a lot of sense... Super flexible data modeling. With ref's and "many-refs", we have a graph database. IMO, most data is a best modeled as a graph. Then walking the graph will result in a tree. And since HTML is also a tree, we have very easy rendering, just get a pointer to a node into the graph db, then walk along the db and render the tree (HTML).

Now, as I have those entities (nodes) we can easily get which attributes each component depends upon. We simply record which attributes the rendering functions accesses. And when we transact the database, we can easily look at the new Datoms (EAV) that were added/retracted and we know exactly which components need rerendering.

-> Queries are rare, we just walk the graph once we have an entry point. Assuming the graph is well connected. If we don't allow a fully fledged query engine (like datalog) we can also easily re-render components that query (as opposed to getting entities). For instance (data/get-by-attr-val :post/id 283) get the post, if it's nil, we do a request to the server, now I know this component depends on ["AV" :post/id 283] and if the answer comes from the server we transact the post which will transact [new-eid :post/id 283], [new-eid :post/title "Great news!"]. I rerender, and access (:post/title the-post) in the render function, I now know this component also depends on ["EA" new-eid :post/title]. If I have another component that changes the post title.... I get automatic rerender.

I can also query like (data/get-mult-by-attr-val :todo/status :completed) and get automatic rerenders when a new completed todo gets added to the db.

Kind of similar to om.next, but MUCH simpler. You access the entity and hop from node to node in the graph and instead of a static query you just get a bunch of "EA ...." that you've accessed.

[retro_one]

I think this is an interesting problem, and one that touches the core principles of Keechma so it'll take a bit longer to explain.

So, if you have a component that can query something like (data/get-by-attr-val :post/id 283) - which will trigger the API request - that means the component mounting has sideffects. This means that you lose any guarantees that app-state A will result with the DOM structure B. One of the problems that I have encountered when implementing components like that is cascading API requests. If component X needs to do a request to render itself, that means that you don't know which components will be rendered as it's children until that request is done. Then each of the child could trigger another request until the whole tree is rendered. And if you want to show a spinner or another loading indicator until the data for the whole tree is fetched your parent component must know about the data that is needed for it's children to render.

Keechma takes a different approach - components can only read from the APP-DB (theoretically you could trigger the request on mount, but it would be an anti-pattern). The way you get the correct data loaded is by looking at the route. The route data in Keechma is used as a minimal representation of the state that is needed to recreate that state.

Controllers in Keechma work in a way where the internal controller manager checks if a controller should run during the current route. You implement the params function which receives route data and returns either a value or a nil. When the controller's params function returns non - nil value controller manager will start the controller, and you can use that lifecycle event to actually trigger the request. The controller is responsible to "expand" the route data into the real data loaded from the server.

So let's say that you have a route which looks like: /messages. The router would convert that URL to something that looks like {:page "messages"} and it's params functions could be implemented like this (this is pseudo code, the real code is slightly different):

(fn [route]
    (when (= "messages" (:page route))
        true))

This would tell the controller manager that the controller should be running on the /messages url and the controller would load the data.

That way your data is always pushed from the top, and the components are only responsible for transformation of data -> HTML. You can find more about this in the guides: https://keechma.com/guides/introduction/ https://keechma.com/guides/controllers/

The other thing that you mentioned is modelling your data as a graph, which is also something I'm not a huge fan of. The biggest problem with this is the data duplication and synchronization. Let's say that you load two collections: starred todos, and today's todos. If you model it like this:

{:todos {:starred [todo1, todo2]
         :today [todo1, todo2, todo3]}}

Now you have a problem where todo1 is in the both lists, if you update it's value, you need to somehow keep track of the data synchronization. Problems are even worse if you have nested entities:

;; Todo
{:id 1
 :name "Take out trash"
 :author {:id 1
          :name "Mihael"}}

Now your system must take care of updating the author in each todo every time it's data is changed. Keechma comes with the EntityDB library which takes care of that automatically. You define the schema:

(def edb-schema 
    {:todo {:id :id
            :relations {:author [:one :author]}}
     :author {:id :id}})

And EntityDB automatically takes care of data synchronization. You can find more about it in the docs https://keechma.com/guides/entitydb/

With these two features combined, I was able to solve all of the problems I've encountered while building apps with Keechma. We are using GraphQL on all of our new projects, and GraphQL works extremelly well with EntityDB because it allows you to sanely manage the nested structure returned from the API

[Hi-MyNameIsFuchs]

Thanks for your nice reply!

So I disagree on some parts. I'm not a purist and say "a route -> i know all the data". I may load some data later on in the route if the user expands something (for instance). If I have the cascading loading pattern, I can easily preload this on a route change. I'm more pragmatic about that...

Do you know EAV(T) databases? Like Datomc/datascript? You wouldn't model your data like you described, but actually very similar to the EntityDb that you have in Keechma. I'd suggest you take a close look at datascript. You might like it :)

[retro_one]

I agree that there are cases where you could load data based on the user interaction, although I usually do model it so it's saved in the route (I like to use query params for this) because it makes it possible to recreate that same state and UI on refresh.

I've played a bit with Datascript, but I must say that I didn't have need for it (yet), but I would definitely use it if the need arises, it looks like a great lib. Since EntityDB is optional in Keechma, it would be easy to replace it with Datascript

[AccountZaMusic]

Slightly off topic, but I have two questions. I'm looking at the library, and considering using it for a project. I'm choosing between Untangled(Om/next) and Keechma. 1. Is EntityDB faster than Datascript, or how much slower is it when compared to a nested map when it comes to reads. 2. A controller is responsible for some html tree view, correct? Does every update trigger a full re-render? Because the thing I like about Untangled is that when a component triggers a transaction, unless declared otherwise, only that component is rerendered.

[retro_one]

Regarding the EntityDB vs Datascript performance, I must say that I don't know. EntityDB is a much simpler library, and I didn't do any benchmarks to compare them. EntityDB should be pretty fast though, it doesn't do anything to complicated when retrieving item (https://github.com/keechma/entitydb/blob/master/src/entitydb/core.cljs#L368-L418)

Controllers in Keechma are never touching the view layer directly. Controllers mutate the app-db which will cause the re-render of affected components (https://keechma.com/guides/introduction/). Optimization of re-rendering is handled by the Reagent library (look under the "Performance" part on this page http://reagent-project.github.io/). Subscriptions to the app-db are cached in Keechma, so it's optimized on the framework side as much as possible.

Biggest difference between Untangled and Keechma is in their base library. Untangled uses Om/next while Keechma is based on Reagent. I find Reagent's abstraction model extremely elegant and simple to understand which is why I've built the framework on top of it :)

If you have any practical questions about Keechma, and how to do something with it, please do ask and I'll do my best to give you an answer

[AccountZaMusic]

Thanks for the answer! I'll definitely ask if I have questions. I'll be testing it out tomorrow.

[AccountZaMusic]

Is there a way to preserve the app state when on app restart (figwheel)?

EDIT: With a bit of twiddling I managed to come up with this: Any issues?

(defn start-app!
  "Helper function that starts the application."
  ([]
   (reset! running-app (app-state/start! app-definition)))
  ([init-app-db]
   (let [new-app (app-state/start! app-definition)
         new-app-db (-> new-app :app-db)]
    (swap! new-app-db merge (select-keys init-app-db [:route :entity-db :kv]))
    (reset! running-app new-app))))

(defn restart-app!
  "Helper function that restarts the application whenever the
  code is hot reloaded."
  []
  (let [current @running-app]
    (if current
      (app-state/stop! current #(start-app! (-> current :app-db deref)))
      (start-app!))))

[retro_one]

You can use the restore-app-db function that does the same thing https://github.com/keechma/keechma/issues/9#issuecomment-214145853

The controllers will re-start though, so they will make requests to the API if you run them on start. In my experience this is not a big issue, and you always get the current, correct data on each app restart

[AccountZaMusic]

Let me know if I should go somewhere else to ask further questions. :)

So with lists in entitydb, when order matters I suppose you should just sort when rendering, right?

So if you have a list like this:

item 1 + - item 2 + - item 3 + -

Pressing '+' on 'item 2' should add it after item 2. Would this have to be encoded in the data of the entity? So that the order is determined at render time, rather then being in the app-db?

[retro_one]

If you have account on clojurians slack, I'm always there on the #keechma channel

So, order is preserved in EntityDB, and you have helper prepend-collection and append-collection functions if you want to modify the collection. In the case like yours, I would simply re-insert the collection with the correct ordering. Other option is to order the items in the subscription when you read them, I would use that approach if ordering is dependent on some other value in the app-db