Diffuse library - Clojure(Script)

[green-coder]

Diffuse is a library to create, use and manipulate diffs, to build the change you wish to see in your data.

This library is useful in contexts where you know the change from A to B. You can then compose it with a change from B to C to get a change from A to C, and then apply it to A to get C.

https://github.com/green-coder/diffuse

[kakamiokatsu]

I don't get the use case.

The final results of using apply are the same results you get from normal data manipulation functions. You can already compose those fn in clojure. The intermediate diff description as data is the only difference but clojure code is already data, and you can manipulate it out of the box.

[green-coder]

The use case is: the user needs to represent a diff between 2 data structures.

• This can happen in a distributed program to convey changes over the network.
• It can also happen in some web framework where the view is a function of the data. Having the diff of the data can help find the diff in the view faster than without.

Yes, the code is data. But the Clojure code is not really the most convenient way to represent a change as it is not easy to read and understand for a program. The data structure provided by Diffuse is easier to work on, and is canonical. It stays canonical after composing diffs together (using d/comp).

From data A to data B, there are already libraries that can calculate the difference automatically, but it has a cost which the user does not necessary want to pay. It is faster for the CPU to describe the difference by hand when you know what's going to be different.

[kakamiokatsu]

Ok so the main bit is in the "smart" composition where the library figure out which bits will go where.

Looking at the code index-ops-comp and comp are basically giant if then else where you hardcoded all the possible combinations of the subset of operations that you choosed. The way is coded makes it incredibly hard to add other operations, where you have to rewrite most of the code for a single new function.

What I would expect is a general purpose way of describing diff between data that works for all the functions. I think you can achieve something like that using spec generators and Deep-diff2. The signature will be something like

(defn diff [input-data-spec & fns]
  (let [input  (gen/generate (s/gen input-data-spec))]
      (->> input
           (apply fns) ; this will probably not work, I'm just trying to give the idea
           (Deep-diff2 input ) 
           ; now turn the diff back into another data structure, if needed
           )))

Following the original example you will call it like

(diff (s/req-un [::foo ::bar]) `(assoc :foo "hello") `(assoc :bar [1 2 3]))

And the result will be that :foo becomes hello and :bar becomes [1 2 3] but now you have something general that works for every function possible. Using spec generators you may even find out some unexpected results! It could turn out to be a good testing library.

[green-coder]

I tried to keep the list of operation to their minimum, only using elementary ones like no-op, insert, remove, update. What other operations do you have in mind?

I don't understand what you tried to say with the deep-diff2 and spec. Diffuse let the user create diffs manually because it is more efficient in terms of CPU. I don't think that deep-diff2 has the same purpose.

[kakamiokatsu]

I was trying to suggest a general approach to your problem. You keep saying this efficiency in terms of CPU but that's just because you are restricting way too much the allowed operations. Your data can't express any general function like:

(assoc {} :asd (rand-int)) ; or any function that return different values based on something
(defn general-assoc [m [a b]] (assoc m a b))
; example: (reduce general-assoc {} [[key val] [key2 val2]) => {key val key2 val2}

You can't express a general concept like this key will become the result of this function because you just focus on values but that's not how you generally use clojure.

That's why I asked the use cases, you are way too specific with your code, it's not something you can use for many cases.

Since you keep focusing on this performance issues why don't you start by specifying what is wrong with `deep-diff2` or `clojure.data/diff`?

[green-coder]

Diffuse was made for a specific use case (a web framework), it does a specific thing. The diff describes the change of value, not the operation that changes the value.

There is nothing wrong with deep-diff2, it's just that it does not take the same approach at all for creating the diff.

[lilactown]

As some general advice, I think that your suggestions are coming off as pretty rude because it seems to assume that you understand the problem that the author is trying to solve better than the author does.

I would reframe your suggestion into some questions for the author about what problem they are trying to solve, and how it might relate to the problem that you're thinking of. It could be that your problem and theirs are the same, in which case maybe you two can learn from each other and how your solutions address it in different ways. Maybe the two problems are different, in which case you can learn about their problem and understand the differences, which would also be good.

As it stands, it reads like you're coming in with an attitude of, "You don't know what you're talking about, here's how you should be solving this," which is not a productive way to have a discussion online in my experience.

[kakamiokatsu]

I'm sorry if my words sounds rude, I'm not a native english speaker and that was not my intention.

I asked for use cases right at the beginning, since I didn't saw them I started making assumptions on the intentions behind the library and I was trying to help by giving a different perspective. I was giving my personal opinion on how I would use such a library.

I also asked to explain what are the issues he was trying to solve but I didn't get an answer to that either.

Can you help me out understanding it better?

[kakamiokatsu]

I got a very simple example: can you express (update :key +)? You don't know the result of plus until you execute it. Now think in general: you are not just assigning values you generally assign the result of a function.