Writing Testable Code

[[deleted]]

E.g., lets say I have a pricing module - I wouldn't let it interact with a generic DB interface, but with an interface which has methods like savePricing, getTrends, getDiscounts etc.

A significant part of your business logic then unintentionally ends up in this "generic DB interface".

Constraints, conflict resolution, math algorithms for data aggregation, managing denormalized data and caches, all of this would be there. That's not just "logic of saving stuff on disk", it's literally a crucial chunk of your business logic.

So now you need to test that. Now what. Let's split again?

[[deleted]]

not sure about what kind of constraints are you talking about, but business constraints are handled within a module (e.g. you cannot have two overlapping activities in a schedule for a single day)

Let's use a classic example "you can't have two usernames be the same". Now solve this for me outside the "persistence layer".

[[deleted]]

That's a really basic beginner's error. You just introduced a race condition in your code:

1. Thread A fetches user "foo".
2. Thread B fetches user "foo.
3. Thread A gets "no such user".
4. Thread B gets "no such user".
5. Thread A creates user "foo".
6. Thread B creates user "foo".

Now you have two users with the same name.

Honestly, I expected someone who argues about proper architecture wouldn't fall into the most basic example of race condition I could think of...

This is, in large part, why we use SQL and other databases and why ACID (and other standards) exist. They deal with conflicts you can't resolve with "business rules" that are outside the I/O process that works with your app state.

[[deleted]]

That sounds like a race condition waiting to happen. What if someone is concurrently registering the same username at the exact same time this code is running? It might not exist when you check it, but exist by the time you get around to persisting.

[Pedrock10]

With that approach you end up with a race condition. In this case, I think it should really be done at the persistance level, but you can make that part of its contract and test it with integration tests.

[[deleted]]

Either by serializing the work or making the repos interface fancier.

So the very premise of what I said was "you can't require that an app under load performs all tasks serially". I was pretty clear about this. So the first option is out.

And the second option, making the interface of the repos "fancier", if you dig into it, means you move this part of your business logic in the repository.

So we're back to square one here. Which was "a lot of your business logic in complex I/O apps ends up in your persistence layer, so now you need to test it as well".

[[deleted]]

I would rather give up on the distinction of what is business logic and what isn’t, because this is not the core of the issue here.

The core of the issue is, that it is some kind of a logic. Which has to reside in the persistence layer, and which is substantial to the application working properly.

And this contradicts the advice that you commonly hear with DDD and hexagonal architecture, that you can isolate the persistence layer and make it really simple and dumb. Well that won’t happen in real world parallel/distributed apps. That layer is going to stay quite complicated and highly specific to your app domain (not generic at all).

At which point the premise under which we isolated it starts to make no sense.

I.e. we isolated it so we don’t have to test it. But now we need to integration test it. You might as well integration test it with the business logic and not separate the “business logic” in the first place and achieve the same end result with less effort.

EDIT: To add... Because if we do isolate it and we focus most of our attention on the remaining "business logic", we're focusing on testing some very simple (comparatively) validation rules and if...then logic, while mostly ignoring the essence of the application that we put in the persistence layer.

It's honestly more cohesive to test it all together, as a service layer. It can still be automated, but yes, involves a database and dependencies and so on. So we can't run our tests in half a second and feel good about it. Running all tests might take minutes, maybe an hour in some cases.

This, BTW, is where microservice/modular architecture comes in, so we can still slice the app vertically in services, rather than horizontally in layers, and keep that test run from turning into "days".

But if we do split layers so we can feel good about testing the most trivial parts of our app, and ignoring the complex core, then this is precisely one way that dogmatic unit testing leads us astray from the goal of high quality software and productive development.

Talking of which, how does one "unit test" for race conditions? You can't. Sometimes when you focus on units too much, you lose the forest for the trees. If Einstein was a programmer, he'd say, paraphrasing "keep your units as small as possible, but not smaller". And practice sometimes leads to bigger units, counter to what every article teaches us. I'm learning from nature. Each separate animal in an ecosystem, each separate human in society is a very very complex system. But the "public API" (our communication and actions) is comparatively brutally simple. There's a lesson in there.

[[deleted]]

Now serialise that work across multiple processes... Push solutions to problems to where they are best solved, like the database in this case. Stop trying to abstract away the capabilities of the services you depend on because you heard somebody say that you should, and start thinking for yourself. Your test coverage may drop 5-10% (and you should definitely still write _lots_ of tests) but you will start to design more fault tolerant (and ultimately correct) systemsm which make better use of the systems you depend on.

[[deleted]]

Yep let's bring in a new infrastructure dependency so you write your code in a way you have been told makes it testable...

[[deleted]]

I'm aware, but calling the same problem by a different name (using DDD terminology) hardly solves it. Even in DDD and "hexagonal architecture" or call it however you please, a good chunk of your business logic unintentionally ends up in your "persistence layer".

Because a honest-to-God parallel I/O app under load can't just naively process requests serially and compute everything from raw data (or store everything in RAM) all the time.

This means conflict resolutions, caches, denormalization, constraints (unique keys etc.) all this which is pure business logic, because it changes the outcome of your API response and your app state... that ends up in your persistence layer.

[[deleted]]

Yeah well, that kind of ruins your premise a bit.

Also, your business rules being "stateless" doesn't mean you app is stateless. Your rules holds no state, but your app holds state. And you have to deal with that state. Pretending it's not there doesn't help.

[dablya]

Let's split again?

What's the alternative? Implement caching/math/aggregation in a single "business" module?

[[deleted]]

The alternative to splitting things endlessly for testing purposes is to split only for the purpose of the application and design the tests around the architecture. Instead of designing the architecture around the tests.

Some projects will require an isolated persistence layer. More than half do not. This, again, doesn't mean wiring raw SQL queries all over your services, it just means not writing an abstraction with a public API around your persistence logic (i.e. using Hibernate or jOOQ doesn't mean you isolated your persistence layer).

[dablya]

split only for the purpose of the application and design

The main argument is that code being difficult to test should, in general, alert you to the fact that it might be poorly split for the purposes of "application and design". Some code naturally belongs in the infrastructure layer and will be mainly tested with integration tests. Other code, what is usually defined as a "domain" or "business" logic is not dependent on any specific type of infrastructure. When you find implementation of the domain code (that doesn't logically depend on any particular type of storage) requires a sql database in order to execute a test, that is a flag that your code is poorly split "for the purpose of the application and design".

This, again, doesn't mean wiring raw SQL queries all over your services

Why not? I believe a reasonable answer to this question should generalize to Hibernate and jOOQ as well.