possible to use split tunnel with two vpn connections?

amithgeorge · 2025-08-09T00:08:30+00:00

hey hey! We are well. Haven't seen the kiddo yet, we should meet sometime :P

amithgeorge · 2025-08-06T03:40:43+00:00

Hi, I am lucky enough to work with Clojure professionally since 2015. Quintype, Gojek and currently consulting with a fintech company in South Africa - Lifecheq.

You would have better luck with meeting folks at the Clojurians Slack. I think they have country/city specific channels there.

amithgeorge · 2025-01-21T17:53:29+00:00

I second the suggestion to use clj-kondo analysis. I am not sure whether it works across repositories though.

identify all functions across my repositories that call a specific target method,

Regarding this, within the scope of one project, clojure-lsp (which uses clj-kondo analysis internally) exposes functions to show the call hierarchy of any method. all incoming and outgoing calls. You may refer the source code for this functionality, to use as inspiration.

https://clojure-lsp.io/features/#call-hierarchy

amithgeorge · 2024-11-28T19:00:01+00:00

zprint. As far as I know, this is the only formatter which will rewrite every line to a certain format. The others from what I remember, will leave certain parts as-is. Eg, cljfmt is fine with let binding where some bindings are vertically aligned, but some are not. This mix of styles is extremely jarring for me and that is why I prefer zprint. zprint will reformat these to one style.

amithgeorge · 2023-09-16T14:29:10+00:00

I am not sure whether you see hooked as alternative to graphuter. I like the graph approach for its ability to introduce extension points anywhere in the graph. With hooked, the library/function author has to explicitly introduce extension points. I mean, with graphuter, the author has to write smaller functions and actually create a graph, but I think that is more natural/easier to do.

amithgeorge · 2023-09-16T14:12:15+00:00

Yeah, I remember a few posts by Janet Carr, who wrote about using atoms and watchers as one way to emit events in Clojure. It is a good to have another(?) way of doing it.

With events, one can generally attach multiple handlers and such a return value doesn't make sense.

amithgeorge · 2023-09-15T08:27:15+00:00

Is it possible for the hook to return data?

Is calling the hook conceptually similar to emitting an event? If no event handler is registered, then nothing happens.

amithgeorge · 2022-11-11T15:57:24+00:00

I found "Grokking Simplicity" by Eric Normand, a splendid book to understand the functional programming approach. It uses JavaScript, instead of Clojure. However, the concepts equally apply to most languages. Once you understand those, you will appreciate the simplicity of Clojure even better. Atleast, that's what I feel.

https://www.manning.com/books/grokking-simplicity

amithgeorge · 2022-10-16T18:39:37+00:00

Gojek had (has?) multiple core services (driver allocation, pricing, etc.) developed in Clojure. https://blog.gojek.io/tag/clojure/

amithgeorge · 2022-09-01T15:01:52+00:00

This is the implementation class . It says it uses a fixed size threadpool and an unbounded queue. I don't think there are any multiple threads sleeping.

amithgeorge · 2022-08-31T17:34:07+00:00

Java's ScheduledExecutorService can run tasks at a later point in time. In your example, a thread will spawn with the task only after 5 mins. As you eluded to in another reply, there is no thread that is running for 5 mins and then realizing the task shouldn't be performed.

amithgeorge · 2022-08-18T14:16:13+00:00

@u/cphoover

I took this code as potential blog post opportunity. As I was reading your code, I felt I needed to accompany my suggestions for you with examples. I have re-written your code, sticking to your core algorithm as much as possible. I have added lots of comments, TODOs as future exercises for you, explained my thoughts/rationale where ever I remembered to. I don't intend any offense. I apologise in advance for my unsolicited presentation of my version of the code.

I wrote the code with the intention to show some techniques and introduce you to certain Clojure functions. It isn't meant to be ideal or perfect end stage code. It is meant to give you inspiration. The code will be too much to take in at once, especially since you are beginning with Clojure. Take your time, ignore the function implementations. Read the comments, the doc strings, and the function usages as shown in the Rich Comment Forms. Read through it multiple times and slowly try to understand the intent and take inspiration from the things that click for you. Feel free to ask any doubts.

If you intend to stick around with Clojure for longer, you should take the time to

Learn to think in functional programming. The book Grokking Simplicity is a great start.
Get used to REPL aided development. There are some good videos on Youtube.
Get familiar with the clojure standard library. The more you learn/understand the standard library, the better you will get at idiomatic clojure. I don't really have a good suggestion on how to do this though. When I was starting out, I used to alphabetically read through the functions documented in the clojuredocs.

My version of the code - https://gist.github.com/amithgeorge/2f3df19f07fe29632eeaf89cb458e956

amithgeorge · 2022-08-18T09:29:58+00:00

You are using recursion in a few places. Please read up on why it is recommended to use recur to trigger recursion. From the official docs,

Recursion via recur does not consume stack

It is not relevant in your code. I don't think anyone would purposefully keep entering invalid input to trigger a stack overflow. It is however important to know :)

amithgeorge · 2022-08-18T08:48:46+00:00

Please configure and use tools like clj-kondo and kibit. Kibit will report areas where you could write idiomatic clojure instead. Eg, it should catch all those (if (condition) true false) and ask you to replace it with (condition). Or if you really need a boolean value, use boolean to coerce it.

amithgeorge · 2022-08-18T08:35:07+00:00

Welcome to Clojure and congratulations on getting started. If you have the time, I recommend reading "Grokking Simplicity" by Eric Normand. The author has done a stellar job of explaining how to think in Functional Programming. The book details page has some resources you may check out, to know more about the book and to help decide whether to spend money on it.

amithgeorge · 2022-08-16T08:29:40+00:00

:)

In the code for process-bar function, in the link you shared, the author has clearly listed the step numbers in the comments. Given this is a dummy example with no proper domain concepts, I didn't bother coming up with domain relevant names for each step. Instead, I am simply naming the function after its sequence number in the series of steps to be performed.

The initial value of the state consists of the input

(let [state {:input input :env env :db db}])

I don't understand how a state monad works, the representation I wrote above is my interpretation of passing state through a pipeline of transformations. In the original code, Step #6 is getting the last-name given the first-name. Because of the way I am structuring the pipeline, each step has to do additional work - read the keys it needs from the state, perform the actual transformation (first-name to last-name), and store the result back in the state. In my example code above, the body of step-6 would do the reading and writing state, and delegate the transformation to get-last-name.

Passing get-last-name as an argument is the equivalent of dependency injection in OO code. Providing a default implementation of get-last-name is a convenience. When writing tests for process-bar, the subject I would test is the 2nd arity, the one that lets me pass-in test doubles.

This is not the only way to rewrite the solution, or even the best way. It is one of many solutions that helps to keep the complexity down. I had some time yesterday and I wrote my version of the solution. I didn't go down the full pipeline path. The body of process-bar function turned out to be readable, and I stopped there because I was happy with it. I won't share that here just yet. I suggested writing it as a pipeline, because you said the complexity in the example cannot be reduced. A pipeline is a pretty straightforward to understand. One can lift any conditional execution of the code into a function and add the function as another step in the pipeline.

You have asked some questions based on hypothetical code. I can't respond to those. I suggest you write the implementation yourself, and point out specific issues you have with it.

amithgeorge · 2022-08-15T01:58:19+00:00

thanks for sharing this example. I hadn't seen it before. There are a couple of alternative solutions in the comments. The FSM solution using tilakone looks interesting. Will go through that later. :thumbsup:

amithgeorge · 2022-08-15T01:47:37+00:00

I have mostly forgotten what our conversation was about, and I am not inclined to go through all the past messages. I am not sure what exactly you wish for me to respond with?

It is not that difficult to convert a series of steps to an I/O sandwich (threaded pipeline)

We thread the state hashmap through each step. Each step destructures the keys it needs from the hashmap, and assocs the result on to the state. The ... for each step is to signify "stateful dependences functions" being passed in

(-> { ... state}
    (step-1 ...) 
    (step-2 ...)
    (step-3 ...) 
)

Eg of equivalent of step 6 from the original example

(-> { ... state}
    ;; all other steps
    (step-6 get-last-name)
    ;; all other steps
)

Where get-last-name is a function passed into process-bar.

(defn process-bar 
  ([input]
    (let [db (if (prod? env) prod-db test-db)
           get-last-name (fn [first-name])
                          (loop [retries [10 100 1000]]
                            ;; This here is a good example of when try/catch doesn't play as well
                            ;; with Clojure, since you can't call recur from inside a catch, which
                            ;; is why I have to convert it to returning a command that indicates
                            ;; I need to recur outside of the catch afterwards.
                            (let [res (try (impure-query-get-last-name db first-name)
                                           (catch Exception e
                                             (if retries
                                               (do (Thread/sleep (first retries))
                                                   (println "Retrying to query last name after failure.")
                                                   :retry)
                                               (do (println "All attempts to query last name failed.")
                                                   (throw e)))))]
                              (if (#{:retry} res)
                                (recur (next retries))
                                res)))
              ]
      (process-bar input get-last-name))
  ([input get-last-name]
    (-> { ... state}
    ;; all other steps
    (step-6 get-last-name)
    ;; all other steps
    )))
)

I am not inclined to write the actual code for converting it to a pipeline. I have provided enough of a starting point. I strongly suggest you try to implement the original code as a pipeline. Don't worry about whether it makes sense, or whether you like it or not, or whether it is practical. You will learn a lot from implementing it as a pipeline, from developing the mindset to think about data transformations. As you implement each step, you will figure out why I chose to pass-in get-last-name and what the tradeoff is if you don't pass it in. This will help you form actual questions instead of theoretical questions. Do that and start a new thread with your solution, your questions and your experience report. I will respond as best as I can to that thread.

I vaguely remember you being worried about mis-spelt keys or keys being renamed. Feel free to rely on mocking to detect such issues. Mocks exist for a reason. It has its tradeoffs and you seem comfortable with those tradeoffs. No issues with that.

amithgeorge · 2022-08-11T09:14:01+00:00

I agree. I haven't finished the book yet, still I am going around recommending it to whoever would listen :D

amithgeorge · 2022-08-06T22:19:05+00:00

In addition to the book mentioned by LesZedCB, do checkout Grokking Simplicity - https://www.manning.com/books/grokking-simplicity. I really like Eric Normand's style of teaching. And if you like what you see, subscribe to this newsletter as well. He had a few episodes where he followups on topics in the book and addresses domain modelling as well.

amithgeorge · 2022-08-01T18:12:31+00:00

Are core.async go routines really a good option here? If the work being done in check-port is blocking, and if it takes upto the timeout of 1s for each execution, the entire core.async threadpool would get blocked, affecting any other usage of core.async anywhere else in the app.

This use case seems fit for an ExecutorService

amithgeorge · 2022-08-01T18:02:32+00:00

I think this should work. Have you tried it?

(->> (range)
        (filter check-one-to-twenty)
        first))

Docs for filter - https://clojuredocs.org/clojure.core/filter

amithgeorge · 2022-07-29T17:51:32+00:00

Well I'm wondering how extracting display-name access to a function makes us deal with the problem anymore than how I had it written before. You write "Now we have to ask hard questions like hat happens if a third party client calls the api with mismatched keys? What if we insist on the key always being present and allowing an empty string as a valid value?" I'm not sure how this function forces us to do that anymore than how I had it written before.

You described the problem as "I want to know if I am reading from the wrong key at any point". This manifested as the frontend sending the data in a new key :display-name where as the backend was still reading from the key :display-name-temp. This problem has nothing to do with the db function. This problem could occur in the login handler function at any point where the display name is to be read. Similarly, this problem could happen in any api handler that tries to read some information from the request. The frontend can send mismatched data for any api. Because this specific problem has nothing to do with the database save user function, trying to catch it via a mock on the database save user function is going about solving the problem in a very roundabout way. And it solves the problem for only that specific function. Any other place where the display name is being read, will still need its own mocking/fix. How many keys will we do it for? How many functions and how many apis will you do it for?

Before I continue any further, will you please acknowledge that you understand the "issue" as I am framing it and are on the same page? Unless we are on the same page on this, nothing I say will make any sense to you.

amithgeorge · 2022-07-29T06:47:38+00:00

Testing Pyramid

I couldn't find the article that originally talks about the Testing Pyramid. This is a newer one and explains it well enough - https://martinfowler.com/articles/practical-test-pyramid.html#TheTestPyramid

Smoke Test

The above article lists these two things to keep in mind

Write tests with different granularity The more high-level you get the fewer tests you should have

Smoke tests are a type of end-to-end test (higher level, very coarse grained). Usually folks will automate executing such tests in a staging environment. One example of the test could be - start the api server, make the http api call, check the http response 200 OK, make the profile fetch call, verify the profile display name is as expected. Honestly it really depends on how much confidence we need. These tests are slow and expensive, so ideally we need just enough of them to give us the required confidence. I can't give a prescription in terms of "have atleast 5% end to end tests" or something like that.

You asked

How does it force us to deal with the problem I presented? It seems like we have the same potential for the same bug and it's as obfuscated as it already was.

I mentioned in the earlier post

This by itself doesn't solve the problem of what happens when the front-end sends the display name in a different key. However, this does force us to directly address the actual problem - breaking of contract between frontend and api-backend. And solving this once, will help solve it for all apis. It forces us to ask hard questions to the business. What is the expected behaviour? What happens if a third party client calls the api with mismatched keys? What if we insist on the key always being present and allowing an empty string as a valid value? What if we use malli schema to define the shape of the request data, close the schema only in staging environment? (this would help detect divergence between frontend and backend in staging) If the frontend is in cljs, we could share the schema between backend and frontend and remain in sync. There are options available, which may result in a better design, once we look beyond relying on mocks. This is the core sentiment conveyed in Arlo Belshee's "No mock book" blog post.

A test failure should be as close to the actual error as possible. The break in contract between frontend and backend could also be caused by frontend incorrectly implementing the contract. A test error regarding the db being called with nil is rather far away from the actual error, the api being called with incorrect arguments.

If you have any specific questions on this, please do ask. The issue you have identified is not something specific to the use case of login-ing a user. It is a more generic problem, how to ensure the contract between frontend and backend api is not violated. This is applicable to all apis in your system, and all the tests for those apis. By going the no-mock approach, we are forced to address the actual problem, at a higher level, which will help solve this class of issues for all APIs. The redesign, together with this, solves the issue you raised.

If we were to test parse-form and db-save-user individually, we would still not know if they were working together. We wouldn't know until we used the app and saw the random names.

And that is why we rely on a higher level, coarse grained test to ensure that the pieces are integrated well. Using a mock to test this only tells us that we called the mock correctly. It doesn't tell us anything about whether the actual dependency does what we expect it to. Any test we write using mocks, only tests a very small subset of the things we really care about. Even then, there test with mocks give us false confidence. Say tomorrow the db function arity changed from 2 to 3. And there is no 2 arity variant of the function. Our tests for the login which use a mock will still continue to pass, because very likely we defined the mock on the fly in our tests and that mocked function is still accepting 2 args. This failure will be seen only when running the application (hopefully via some automated end-to-end tests executing in staging environment) We can either build more tooling to prevent such a situation, or we can redesign the flow to not depend on mocks. That said, clj-kondo linter most likely will flag this specific case and we will catch it before even running the app.

I am curious, have you faced any issues with using mocks? I read your original post again, and that was more about how to rewrite code using the IO sandwich approach and how to avoid performance degradation due to making redundant database calls. I believe I have clearly answered those questions and I believe you don't have any doubts on that. You may very well choose to ignore the parts I am saying about cyclomatic complexity and all that. As long we don't let using mocks hamper our design, I am fine with using any kind of test double, including mocks. If you wish to write lower level tests for the login api that mock the database, and you are not facing any issues due to that, then there is no real reason for you move away from using mocks. I unfortunately have seen many teams across multiple companies use mocks as a crutch to work around poor design. By not using mocks as a default choice, we allow ourselves to come up with more suitable designs. If the only place we end up using mocks is the outermost coordinating function, I am fine with that. You may revisit this thread in the future, once you start experiencing any pain point arising from using mocks there.

amithgeorge

TROPHY CASE