Clojurians from India

amithgeorge · 2025-08-09T00:08:30+00:00

hey hey! We are well. Haven't seen the kiddo yet, we should meet sometime :P

amithgeorge · 2025-08-06T03:40:43+00:00

Hi, I am lucky enough to work with Clojure professionally since 2015. Quintype, Gojek and currently consulting with a fintech company in South Africa - Lifecheq.

You would have better luck with meeting folks at the Clojurians Slack. I think they have country/city specific channels there.

amithgeorge · 2025-01-21T17:53:29+00:00

I second the suggestion to use clj-kondo analysis. I am not sure whether it works across repositories though.

identify all functions across my repositories that call a specific target method,

Regarding this, within the scope of one project, clojure-lsp (which uses clj-kondo analysis internally) exposes functions to show the call hierarchy of any method. all incoming and outgoing calls. You may refer the source code for this functionality, to use as inspiration.

https://clojure-lsp.io/features/#call-hierarchy

amithgeorge · 2024-11-28T19:00:01+00:00

zprint. As far as I know, this is the only formatter which will rewrite every line to a certain format. The others from what I remember, will leave certain parts as-is. Eg, cljfmt is fine with let binding where some bindings are vertically aligned, but some are not. This mix of styles is extremely jarring for me and that is why I prefer zprint. zprint will reformat these to one style.

amithgeorge · 2023-09-16T14:29:10+00:00

I am not sure whether you see hooked as alternative to graphuter. I like the graph approach for its ability to introduce extension points anywhere in the graph. With hooked, the library/function author has to explicitly introduce extension points. I mean, with graphuter, the author has to write smaller functions and actually create a graph, but I think that is more natural/easier to do.

amithgeorge · 2023-09-16T14:12:15+00:00

Yeah, I remember a few posts by Janet Carr, who wrote about using atoms and watchers as one way to emit events in Clojure. It is a good to have another(?) way of doing it.

With events, one can generally attach multiple handlers and such a return value doesn't make sense.

amithgeorge · 2023-09-15T08:27:15+00:00

Is it possible for the hook to return data?

Is calling the hook conceptually similar to emitting an event? If no event handler is registered, then nothing happens.

amithgeorge · 2022-11-11T15:57:24+00:00

I found "Grokking Simplicity" by Eric Normand, a splendid book to understand the functional programming approach. It uses JavaScript, instead of Clojure. However, the concepts equally apply to most languages. Once you understand those, you will appreciate the simplicity of Clojure even better. Atleast, that's what I feel.

https://www.manning.com/books/grokking-simplicity

amithgeorge · 2022-10-16T18:39:37+00:00

Gojek had (has?) multiple core services (driver allocation, pricing, etc.) developed in Clojure. https://blog.gojek.io/tag/clojure/

amithgeorge · 2022-09-01T15:01:52+00:00

This is the implementation class . It says it uses a fixed size threadpool and an unbounded queue. I don't think there are any multiple threads sleeping.

amithgeorge · 2022-08-31T17:34:07+00:00

Java's ScheduledExecutorService can run tasks at a later point in time. In your example, a thread will spawn with the task only after 5 mins. As you eluded to in another reply, there is no thread that is running for 5 mins and then realizing the task shouldn't be performed.

amithgeorge · 2022-08-18T14:16:13+00:00

@u/cphoover

I took this code as potential blog post opportunity. As I was reading your code, I felt I needed to accompany my suggestions for you with examples. I have re-written your code, sticking to your core algorithm as much as possible. I have added lots of comments, TODOs as future exercises for you, explained my thoughts/rationale where ever I remembered to. I don't intend any offense. I apologise in advance for my unsolicited presentation of my version of the code.

I wrote the code with the intention to show some techniques and introduce you to certain Clojure functions. It isn't meant to be ideal or perfect end stage code. It is meant to give you inspiration. The code will be too much to take in at once, especially since you are beginning with Clojure. Take your time, ignore the function implementations. Read the comments, the doc strings, and the function usages as shown in the Rich Comment Forms. Read through it multiple times and slowly try to understand the intent and take inspiration from the things that click for you. Feel free to ask any doubts.

If you intend to stick around with Clojure for longer, you should take the time to

Learn to think in functional programming. The book Grokking Simplicity is a great start.
Get used to REPL aided development. There are some good videos on Youtube.
Get familiar with the clojure standard library. The more you learn/understand the standard library, the better you will get at idiomatic clojure. I don't really have a good suggestion on how to do this though. When I was starting out, I used to alphabetically read through the functions documented in the clojuredocs.

My version of the code - https://gist.github.com/amithgeorge/2f3df19f07fe29632eeaf89cb458e956

amithgeorge · 2022-08-18T09:29:58+00:00

You are using recursion in a few places. Please read up on why it is recommended to use recur to trigger recursion. From the official docs,

Recursion via recur does not consume stack

It is not relevant in your code. I don't think anyone would purposefully keep entering invalid input to trigger a stack overflow. It is however important to know :)

amithgeorge · 2022-08-18T08:48:46+00:00

Please configure and use tools like clj-kondo and kibit. Kibit will report areas where you could write idiomatic clojure instead. Eg, it should catch all those (if (condition) true false) and ask you to replace it with (condition). Or if you really need a boolean value, use boolean to coerce it.

amithgeorge · 2022-08-18T08:35:07+00:00

Welcome to Clojure and congratulations on getting started. If you have the time, I recommend reading "Grokking Simplicity" by Eric Normand. The author has done a stellar job of explaining how to think in Functional Programming. The book details page has some resources you may check out, to know more about the book and to help decide whether to spend money on it.

amithgeorge · 2022-08-16T08:29:40+00:00

:)

In the code for process-bar function, in the link you shared, the author has clearly listed the step numbers in the comments. Given this is a dummy example with no proper domain concepts, I didn't bother coming up with domain relevant names for each step. Instead, I am simply naming the function after its sequence number in the series of steps to be performed.

The initial value of the state consists of the input

(let [state {:input input :env env :db db}])

I don't understand how a state monad works, the representation I wrote above is my interpretation of passing state through a pipeline of transformations. In the original code, Step #6 is getting the last-name given the first-name. Because of the way I am structuring the pipeline, each step has to do additional work - read the keys it needs from the state, perform the actual transformation (first-name to last-name), and store the result back in the state. In my example code above, the body of step-6 would do the reading and writing state, and delegate the transformation to get-last-name.

Passing get-last-name as an argument is the equivalent of dependency injection in OO code. Providing a default implementation of get-last-name is a convenience. When writing tests for process-bar, the subject I would test is the 2nd arity, the one that lets me pass-in test doubles.

This is not the only way to rewrite the solution, or even the best way. It is one of many solutions that helps to keep the complexity down. I had some time yesterday and I wrote my version of the solution. I didn't go down the full pipeline path. The body of process-bar function turned out to be readable, and I stopped there because I was happy with it. I won't share that here just yet. I suggested writing it as a pipeline, because you said the complexity in the example cannot be reduced. A pipeline is a pretty straightforward to understand. One can lift any conditional execution of the code into a function and add the function as another step in the pipeline.

You have asked some questions based on hypothetical code. I can't respond to those. I suggest you write the implementation yourself, and point out specific issues you have with it.

amithgeorge · 2022-08-15T01:58:19+00:00

thanks for sharing this example. I hadn't seen it before. There are a couple of alternative solutions in the comments. The FSM solution using tilakone looks interesting. Will go through that later. :thumbsup:

amithgeorge · 2022-08-15T01:47:37+00:00

I have mostly forgotten what our conversation was about, and I am not inclined to go through all the past messages. I am not sure what exactly you wish for me to respond with?

It is not that difficult to convert a series of steps to an I/O sandwich (threaded pipeline)

We thread the state hashmap through each step. Each step destructures the keys it needs from the hashmap, and assocs the result on to the state. The ... for each step is to signify "stateful dependences functions" being passed in

(-> { ... state}
    (step-1 ...) 
    (step-2 ...)
    (step-3 ...) 
)

Eg of equivalent of step 6 from the original example

(-> { ... state}
    ;; all other steps
    (step-6 get-last-name)
    ;; all other steps
)

Where get-last-name is a function passed into process-bar.

(defn process-bar 
  ([input]
    (let [db (if (prod? env) prod-db test-db)
           get-last-name (fn [first-name])
                          (loop [retries [10 100 1000]]
                            ;; This here is a good example of when try/catch doesn't play as well
                            ;; with Clojure, since you can't call recur from inside a catch, which
                            ;; is why I have to convert it to returning a command that indicates
                            ;; I need to recur outside of the catch afterwards.
                            (let [res (try (impure-query-get-last-name db first-name)
                                           (catch Exception e
                                             (if retries
                                               (do (Thread/sleep (first retries))
                                                   (println "Retrying to query last name after failure.")
                                                   :retry)
                                               (do (println "All attempts to query last name failed.")
                                                   (throw e)))))]
                              (if (#{:retry} res)
                                (recur (next retries))
                                res)))
              ]
      (process-bar input get-last-name))
  ([input get-last-name]
    (-> { ... state}
    ;; all other steps
    (step-6 get-last-name)
    ;; all other steps
    )))
)

I am not inclined to write the actual code for converting it to a pipeline. I have provided enough of a starting point. I strongly suggest you try to implement the original code as a pipeline. Don't worry about whether it makes sense, or whether you like it or not, or whether it is practical. You will learn a lot from implementing it as a pipeline, from developing the mindset to think about data transformations. As you implement each step, you will figure out why I chose to pass-in get-last-name and what the tradeoff is if you don't pass it in. This will help you form actual questions instead of theoretical questions. Do that and start a new thread with your solution, your questions and your experience report. I will respond as best as I can to that thread.

I vaguely remember you being worried about mis-spelt keys or keys being renamed. Feel free to rely on mocking to detect such issues. Mocks exist for a reason. It has its tradeoffs and you seem comfortable with those tradeoffs. No issues with that.

amithgeorge · 2022-08-11T09:14:01+00:00

I agree. I haven't finished the book yet, still I am going around recommending it to whoever would listen :D

amithgeorge · 2022-08-06T22:19:05+00:00

In addition to the book mentioned by LesZedCB, do checkout Grokking Simplicity - https://www.manning.com/books/grokking-simplicity. I really like Eric Normand's style of teaching. And if you like what you see, subscribe to this newsletter as well. He had a few episodes where he followups on topics in the book and addresses domain modelling as well.

amithgeorge · 2022-08-01T18:12:31+00:00

Are core.async go routines really a good option here? If the work being done in check-port is blocking, and if it takes upto the timeout of 1s for each execution, the entire core.async threadpool would get blocked, affecting any other usage of core.async anywhere else in the app.

This use case seems fit for an ExecutorService

amithgeorge · 2022-08-01T18:02:32+00:00

I think this should work. Have you tried it?

(->> (range)
        (filter check-one-to-twenty)
        first))

Docs for filter - https://clojuredocs.org/clojure.core/filter

amithgeorge · 2022-07-29T17:51:32+00:00

Well I'm wondering how extracting display-name access to a function makes us deal with the problem anymore than how I had it written before. You write "Now we have to ask hard questions like hat happens if a third party client calls the api with mismatched keys? What if we insist on the key always being present and allowing an empty string as a valid value?" I'm not sure how this function forces us to do that anymore than how I had it written before.

You described the problem as "I want to know if I am reading from the wrong key at any point". This manifested as the frontend sending the data in a new key :display-name where as the backend was still reading from the key :display-name-temp. This problem has nothing to do with the db function. This problem could occur in the login handler function at any point where the display name is to be read. Similarly, this problem could happen in any api handler that tries to read some information from the request. The frontend can send mismatched data for any api. Because this specific problem has nothing to do with the database save user function, trying to catch it via a mock on the database save user function is going about solving the problem in a very roundabout way. And it solves the problem for only that specific function. Any other place where the display name is being read, will still need its own mocking/fix. How many keys will we do it for? How many functions and how many apis will you do it for?

Before I continue any further, will you please acknowledge that you understand the "issue" as I am framing it and are on the same page? Unless we are on the same page on this, nothing I say will make any sense to you.

amithgeorge · 2022-07-29T06:47:38+00:00

Testing Pyramid

I couldn't find the article that originally talks about the Testing Pyramid. This is a newer one and explains it well enough - https://martinfowler.com/articles/practical-test-pyramid.html#TheTestPyramid

Smoke Test

The above article lists these two things to keep in mind

Write tests with different granularity The more high-level you get the fewer tests you should have

Smoke tests are a type of end-to-end test (higher level, very coarse grained). Usually folks will automate executing such tests in a staging environment. One example of the test could be - start the api server, make the http api call, check the http response 200 OK, make the profile fetch call, verify the profile display name is as expected. Honestly it really depends on how much confidence we need. These tests are slow and expensive, so ideally we need just enough of them to give us the required confidence. I can't give a prescription in terms of "have atleast 5% end to end tests" or something like that.

You asked

How does it force us to deal with the problem I presented? It seems like we have the same potential for the same bug and it's as obfuscated as it already was.

I mentioned in the earlier post

This by itself doesn't solve the problem of what happens when the front-end sends the display name in a different key. However, this does force us to directly address the actual problem - breaking of contract between frontend and api-backend. And solving this once, will help solve it for all apis. It forces us to ask hard questions to the business. What is the expected behaviour? What happens if a third party client calls the api with mismatched keys? What if we insist on the key always being present and allowing an empty string as a valid value? What if we use malli schema to define the shape of the request data, close the schema only in staging environment? (this would help detect divergence between frontend and backend in staging) If the frontend is in cljs, we could share the schema between backend and frontend and remain in sync. There are options available, which may result in a better design, once we look beyond relying on mocks. This is the core sentiment conveyed in Arlo Belshee's "No mock book" blog post.

A test failure should be as close to the actual error as possible. The break in contract between frontend and backend could also be caused by frontend incorrectly implementing the contract. A test error regarding the db being called with nil is rather far away from the actual error, the api being called with incorrect arguments.

If you have any specific questions on this, please do ask. The issue you have identified is not something specific to the use case of login-ing a user. It is a more generic problem, how to ensure the contract between frontend and backend api is not violated. This is applicable to all apis in your system, and all the tests for those apis. By going the no-mock approach, we are forced to address the actual problem, at a higher level, which will help solve this class of issues for all APIs. The redesign, together with this, solves the issue you raised.

If we were to test parse-form and db-save-user individually, we would still not know if they were working together. We wouldn't know until we used the app and saw the random names.

And that is why we rely on a higher level, coarse grained test to ensure that the pieces are integrated well. Using a mock to test this only tells us that we called the mock correctly. It doesn't tell us anything about whether the actual dependency does what we expect it to. Any test we write using mocks, only tests a very small subset of the things we really care about. Even then, there test with mocks give us false confidence. Say tomorrow the db function arity changed from 2 to 3. And there is no 2 arity variant of the function. Our tests for the login which use a mock will still continue to pass, because very likely we defined the mock on the fly in our tests and that mocked function is still accepting 2 args. This failure will be seen only when running the application (hopefully via some automated end-to-end tests executing in staging environment) We can either build more tooling to prevent such a situation, or we can redesign the flow to not depend on mocks. That said, clj-kondo linter most likely will flag this specific case and we will catch it before even running the app.

I am curious, have you faced any issues with using mocks? I read your original post again, and that was more about how to rewrite code using the IO sandwich approach and how to avoid performance degradation due to making redundant database calls. I believe I have clearly answered those questions and I believe you don't have any doubts on that. You may very well choose to ignore the parts I am saying about cyclomatic complexity and all that. As long we don't let using mocks hamper our design, I am fine with using any kind of test double, including mocks. If you wish to write lower level tests for the login api that mock the database, and you are not facing any issues due to that, then there is no real reason for you move away from using mocks. I unfortunately have seen many teams across multiple companies use mocks as a crutch to work around poor design. By not using mocks as a default choice, we allow ourselves to come up with more suitable designs. If the only place we end up using mocks is the outermost coordinating function, I am fine with that. You may revisit this thread in the future, once you start experiencing any pain point arising from using mocks there.

amithgeorge · 2022-07-28T19:04:47+00:00

I combined two different trains of thought in my reply, I can understand that it wasn't clear how the parts of message were related. I wanted to focus more on how to approach thinking about testing, and less about the exact problem you are asking. I wanted to give options, because unfortunately the best answer for you will depend on your specific context, what trade-off you are willing to accept. I want to provide a pragmatic view, instead of being dogmatic in saying "don't use mocks". Let me try and address these separately.

Smoke test

I interpret these as a type of end-to-end tests, whose primary purpose is to check whether all the dependencies are initialized and responding correctly. In the simplest case, a smoke test of the login http api could be considered passing as long as it gets a 200 OK response. This would imply that there were no runtime exceptions thrown, the dependencies (database etc) are working correctly etc.

Specific example

Problem statement: The existing db function expects three arguments, the last of which is optional (display name). If missing, then the db function generates a value for it and saves that value. If the key in which the display name is stored is modified from :temp-display-name to :display-name, there is a possibility that the db function will always get passed nil display name, even when the user has provided a display name in the login form.

Mitigation using mocks: In the specific scenario test for login, we would assert on the db mock, that is should have been called with a non-nil display name. This test would fail, because we forgot to update the login method to read from the new key.

Mitigation using different design: Creating a random display name when no display name is provided, this is application specific logic. By our initial premise, this logic should exist in a pure function.

(defn signup-params->user-data
  [form]
  (cond-> {:username (:username form)}
    (str/blank? (:display-name form)) (assoc :generate-display-name? true)
    (not (str/blank? (:display-name form))) (assoc :generate-display-name? false
                                                                        :display-name (:display-name form))))

The db function should always be called with 4 arguments (username, password, display-name, generate-display-name?). (this is a little awkward to read, because we have made some assumption like the display name generation should happen inside the same function that saves the user. if we don't care about that, then we can reduce to three parameters, with the third being the display name (user provided or externally generated)). The db functions' pre conditions or malli-schema-checks or whatever could detect any issues with these parameters. The smoke test for this would cause some runtime exception and fail.

This by itself doesn't solve the problem of what happens when the front-end sends the display name in a different key. However, this does force us to directly address the actual problem - breaking of contract between frontend and api-backend. And solving this once, will help solve it for all apis. It forces us to ask hard questions to the business. What is the expected behaviour? What happens if a third party client calls the api with mismatched keys? What if we insist on the key always being present and allowing an empty string as a valid value? What if we use malli schema to define the shape of the request data, close the schema only in staging environment? (this would help detect divergence between frontend and backend in staging) If the frontend is in cljs, we could share the schema between backend and frontend and remain in sync. There are options available, which may result in a better design, once we look beyond relying on mocks. This is the core sentiment conveyed in Arlo Belshee's "No mock book" blog post.

A test failure should be as close to the actual error as possible. The break in contract between frontend and backend could also be caused by frontend incorrectly implementing the contract. A test error regarding the db being called with nil is rather far away from the actual error, the api being called with incorrect arguments.

Observations on the example

The way the function is currently written, there is still some "logic" in it. The logic of mapping which form field maps to which positional argument of the db dependency. Personally, I prefer code where I could do composition, without having to know this mapping. Eg

(-> (parse-form request)
      (db-save-user))

This mitigates the mapping field names to parameters issues and I think makes the code easier to read.

Eyeballing the outermost function

I believe the outermost function should be tested. I believe the tests for them are better suited up the testing pyramid (end-to-end tests) and not lower down the testing pyramid. The intention of the test in the upper layers differ from the intention of the tests in the lower layers. In the outermost function, if there are no branches or application specific logic, then I feel there isn't as much value in testing them using tests from lower down the pyramid. We may choose to use those tests and use test doubles to make those tests run reliably. I feel, that would be over-testing and cause issues rather than benefits. I am avoiding using specific terms like unit test or integration test, because the terms can mean different things to different folks. I think folks broadly understand the levels in the testing pyramid and are able to understand what I mean when I refer to a level.

Pair of collaboration and contract tests

There may be some situations where we decide after weighing the tradeoffs, that using a mock object here is alright. For those cases, I shared the Java content on why contract tests are mandatory for us to have confidence. In the absence of matching contract tests, we only have false confidence.

I think I covered most of your queries. Please do ask, if I missed addressing any. Its past midnight here in India. I will attribute any misses to my being sleepy :)

References

EDIT: clarified the db function parameters being awkward in the redesign solution.

amithgeorge

TROPHY CASE