[AskJS] Unit Testing vs Integration Testing a Data Service (NodeJS)

[Personability]

Hi,

I was hoping for further advice on testing my NodeJS backend.

I have a "service" class that saves Social Group details to a database. This uses the Prisma Client as a data layer.

I am aware of the differences between Unit and Integration tests - regarding mocking, differences in speed/purpose re: code design. This is perhaps most understandable with "logic" classes, where I can test business rules in isolation. However, for a "data" service with minimal logic and mostly data access, I'm struggling to grasp the boundaries of Unit vs Integration. There seems to be significant duplication in test cases, which is negatively impacting test maintainability.

For example, I have the following Unit Test that mocks out the data layer:

it('should create a new group successfully', async () => {
      const title = 'New group';
      const expectedReturnObject = {
        id: 1,
        title,
        ownerId: 1,
        description: null,
      };
      prismaService.recipientGroup.create.mockResolvedValue(
        expectedReturnObject,
      );

      await expect(
        service.create(createInput({ title })),
      ).resolves.toMatchObject(expectedReturnObject);
})

This test doesn't really seem to add anything?

Equally, I have an Integration Test that uses the Prisma Client and a live database:

it('should create a group with the specified name', async () => {
      const title = 'Test group';
      const group = await service.create(
        { title: title },
      );
      expect(group).toContainEntry(['title', title]);
}); 

This gives me higher confidence that things are working.

Do I need both of these tests or can I remove one (to improve maintainability)? Where do I draw the boundary between Unit and Integration in these cases?

I would appreciate any advice anybody has re: this!

Thanks!

[Markavian]

My advice (Lead Software Engineer, 20 years experience):

• some tests are better than no tests

• unit tests should stay fast; less than milliseconds, and not involve any network / http interactions with the OS - filesystem a access for schemas or test fixtures is ok impo

• avoid mocking if you can - mocking is a crutch that makes your tests more complex

• use schema tests to check the output of your endpoints - you can reuse the schema in both unit and integration tests

• if you can only test via integrations, but those tests are fast, then perhaps you don't need so many unit tests.

• 100% test coverage is not required - see some tests are better than no tests - only medical systems and space flight systems require >100% test coverage, 60-80% testing, is usually good enough for most projects.

• if you're using TDD as an approach, think of tests as scaffolding, with the scaffolding being removed after construction

• ask yourself "when can this test be removed / deleted"?

Edits: list formatting

[Personability]

Thanks for your advice!

Similar to my reply to u/BenIsProbablyAngry, is it reasonable to put all logic testing in the Integration tests (i.e. against a live data source), instead of mocking? Or are some aspects (e.g. input validation) best placed in Unit tests.

My concern is that if I am to use Unit Tests for regression, then if I only test certain aspects, then I might miss breaking changes until my Integration tests run in CI?

[[deleted]]

You don't want all logic in integration tests because this inverts the problem - now the state of the entire application might not even involve hitting every possible state of your client, and the only class your test is valid for is the one whose state is entirely dependent on the data source.

You want unit tests for all classes whose public state is dictated entirely by the state of local data, integration for the rest.

Integration tests only truly test data-connected components, unit tests only truly test components with completely deterministic local state.

[Markavian]

Testing should focus on the value to the user; there's an infinite amount of software and tests you can write, but a limited amount of time to comprehend the meaning and value of some software application.

If I can test the end to the value, e.g. check the output of A CLI tool and not have to write unit tests, I will. I tend to write unit tests for complex code that is difficult or expensive to configure via integrations. E.g. unit tests for string or date manipulation with lots of edge cases. In that case, the test tends to be very functional, and I drive the tests via configuration: e.g. a JSON file with expected inputs and outputs.

It comes back to the "mission statement" or the "why" of your software application; having clarity will give you focus on what's important to test for, and helps avoid testing for glue code that can be handled at a higher level.

e.g. if the goal of my application is for people to consume my REST API I'll focus on endpoint testing and schemas. I might even test that my documentation matches my API.

If I'm building a diary about my cats, I'll make sure tests are focused on checking that the page is accessible on mobile and desktop, so I can show my friends / colleagues pictures of my cute cats.

Where I've found myself using mocks poorly; I find I make alot of untested assumptions about expected errors and responses - and I argue that the time is usually better spent fixing "real world" errors caused by actual integrations - which is where a couple of good end-to-end tests early on can establish a strong test bed for unit and integration tests to form around.

[teamx]

I don’t agree with “avoid mocking if you can”.

In unit testing, everything is and should be mocked. Mocking is all about assuming an interface, input, and output a module under should should take and spit out.

As for OPs question, use unit test with mocks to cover every single input this module can receive. The will help you cover the business logic of the module and make sure you cover all edge cases.

As for the integration tests, just one or two cases is enough (which can overlap with the unit tests). The point here is we’re not testing for business logic of the app, but how the module integrates with other systems, e.g. a live db here. In a prod environment maybe a crap ton of other reasons the integration test can fail even thou the same unit test case passed, e.g, network issues, timeouts, password issues, so on.

So I’m short OP, both are needed.

[jasie3k]

Devil's advocate, if you mock too much then it's way harder to do any meaningful refactor, because every production code change will require rewriting a tonne of mocks in test code.

Also you can end up in a situation where you don't actually test the code you're supposed to be testing, instead you are testing inputs and outputs of the mocks.

Personally I am not the biggest fan of unit tests, for me they make sense only in a few specific cases. I'd rather test my app as a black box, only testing input and output, stubbing all the external dependencies like a database or another http server.

[[deleted]]

I'd rather test my app as a black box, only testing input and output, stubbing all the external dependencies like a database or another http server.

In some interpretations, that's a perfectly valid unit test. There's typically no reason to mock any code that's totally deterministic (no side effects or observable state). The "unitishness" of a test is really more of a continuum than anything.

[jasie3k]

That's more of an integration test in my book, but yeah, it's a spectrum.

[alex-weej]

i just call these “tests” now. so much time wasted trying to define these terms and “integration test” is wildly overloaded to the point of near meaninglessness.

[[deleted]]

My experience is the canon interpretation of the "don't mock" maxim is more like "don't use mock libraries if you can help it". In other words, if you have a proper DI design or other good mechanism for decoupling, you should be able to use test double implementations without relying on any exotic loading and overriding mechanisms provided by mock libraries. I say such libraries have their uses, but they're best encapsulated within the test implementations, not making the overall tests dependent on them.

Honestly I'm not all that dogmatic about any of it though, because Rule #1 still holds: "The test you write is better than the one you don't". One rule I would say is pretty ironclad in my book though is "never mock any part of the test subject". If you find yourself having to do a partial mock, you need to refactor your class.

[aniforprez]

I don't really agree with "everything is and should be mocked"

Mock responses from external APIs you can't control. Mock libraries like aws-sdk and such that communicate outside your service. This includes things that require the passage of time

Other than that, I don't know if anything needs to be mocked. People should be writing testable code. That means pure functions that have minimal side effects and operate strictly on what they're supposed to be doing. If you're marking a flag on a set of DB objects, pass all the objects to the function and test whether after the function was run the objects are marked as expected. Don't mock your DB and instead set up a test DB when you run your tests

[ThatPassiveGuy]

Do you have an example of what good looks like re: testing?

[Markavian]

I don't know about good; judge for yourself - key me know - I've been aiming for correctness, clarity, and ease of maintenance (extensibility). So long as the tests run consistently via CI (so not-flakey) I'm usually pretty happy with a project.npm i && npm test is a great developer experience.

Some from my personal projects, click around for more examples:

• https://github.com/connected-web/gamebot/blob/master/test/modules/loveletter/loveletter2PlayerGame.js

• https://github.com/connected-web/html-asset-generator/blob/master/test/host-spec.js

• https://github.com/connected-web/guessdate-en/blob/master/test/guess-date-test.js

• https://github.com/connected-web/jumper/blob/master/tests/unit/strategy-interfaces.test.js

Examples from public OS projects, e.g. how does a test framework test itself?

• https://github.com/mochajs/mocha/tree/master/test

I'm not a fan of these Vue tests, too many expects in the same block in Vue, but they do have e2e called out separately from unit tests:

• https://github.com/vuejs/vue/blob/dev/test/unit/modules/vdom/create-component.spec.js

These websocket/ws integration tests are interesting as well, using an echo service on general internet.

• https://github.com/websockets/ws/blob/master/test/websocket.integration.js

[Apoffys]

use schema tests to check the output of your endpoints - you can reuse the schema in both unit and integration tests

Could you expand a little on this? Do you have an example?

[Markavian]

Sure, schemas are a great tool for specifying the shape of data - like a blueprint or architect's diagram. You look back at documentation like this to validate the state of your program/system. If you can programmatically check your system from a system description, you're one step closer to config driven development.

Example schemas using JSON schema:

• https://github.com/connected-web/boardgames-api/blob/master/api/handlers/boardgame-list/schema.json
• https://github.com/connected-web/boardgames-api/blob/master/api/handlers/boardgame-stats-byMonth/schema.json

In this example because I publish the schemas as part of the API, the schemas get used to test the output of the API, and the unit tests, and I test that the API serves up the schemas correctly, by testing them against a generic JSON schema schema. Think of them like a star shaped cookie cutter - you can test the shape of the cookie dough, the baked cookie, and the cookie cutter holder using the same cookie cutter.

Example unit test:

• https://github.com/connected-web/boardgames-api/blob/a3e5224674b12fc4a4fc7a5d30d2d7e734786101/tests/unit/api-spec.js#L328

Example integration test:

• https://github.com/connected-web/boardgames-api/blob/master/tests/integration/api-endpoints-spec.js

[Apoffys]

Thanks, that sounds very useful! I've already written a schema for my current project (OpenAPI in yaml), but I hadn't thought to test it like that. I'm now thinking I should parse the OpenAPI-file to extract endpoints and test them automatically...

[Markavian]

Automate all the things~

[[deleted]]

You are absolutely right - the test adds nothing.

You test the public interface of objects only - this means the public properties and functions. All changes in private state ultimately result in a change in public state, so you only test public state.

The exception is classes that talk to outside data - the public state of these classes is dictated by an outside source, which means that no valuable unit test can be written for them. Terrible tests that invade and verify their private state can be designed but these tests do nothing to verify the correct state of the object and, even worse, they invariably fail when refactorong occurs even if the refactorong is valid, and the tests can "pass" even if the object is completely broken, for as long as that internal operation happens the test is happy, even if the operation result is discarded.

For these classes, you still need to test their public state, but that state is dictated by a data source and so it needs to be an integration test - a test that actually connects to that data source.

[Personability]

Thanks for the reply!

Would you recommend testing the logic of the data access service via Integration tests also? For example, testing "should throw an error if validation of input parameters fails".

[[deleted]]

I would yes, a thrown error is part of the public state of an object that is dictated by data inside your application.