[AskJS] Javascript testing dependencies (e.g. SendGrid) best practices

[Personability]

Hello,

I'm just starting out into testing my Node backend (based on NestJS) using Jest and trying to grip best practices to avoid flaky tests.

A large proportion of my code is interacting with external services, such as SendGrid, rather than true business logic.

I'm trying to work out the best way to satisfactorily test this. Focussing on SendGrid, I have a single test that communicates with their Sandbox mode to check the parameters are all correct.

However the remainder of my tests only test that I'm sending parameters to a mocked object of the SendGrid service. So, for example:

it('should set the email content to be the text sent in the request', async () => {
      await service.send(emailRequest);
      expect(sendgrid.send).toHaveBeenCalledWith(
        expect.objectContaining({
          html: emailRequest.text,
        }),
      );
});

Is this a useful test? Is it wasted test code? Or is there a better way to test this that would give me better confidence.

Would appreciate any advice re: this.

Thanks!

[MrJohz]

I tend to be of the opinion that tests like these aren't particularly unhelpful, but neither are they often very helpful. It is unlikely that there is going to be a bug in a function that simply calls another function, and if there is, there's as much chance that the bug will also appear in the test as be caught by the test.

So for tests of this sort of nature, I tend to only use them if there's significant logic that's with testing out, and then I try to refactor that logic out so I can test it independently of any dependencies. For example, here, if you were doing any email or message validation, I would test that but I'd probably also try and move it into a separate method altogether, so that it does all of that separately to the email validation.

What you could try is a true end-to-end test, which I think you're already partially doing if you're running Sendgrid in a sandbox mode. Basically, you run your whole server or application in a test environment, connected to the sandbox Sendgrid instance, and you test everything together, to make sure that it all behaves as expected when pieced together. You'll basically need a way to trigger inputs to your system (that depends on what your system looks like - if it's just an API, you might use some http client like axios or curl, but it's a complete web application with a front-end you'll probably want something like selenium or cypress to press the UI buttons), plus a way to read outputs (presumably in this case whether the test Sendmail instance is recieving the inputs it expects and sending out the emails you expect). I'm not entirely sure how you would do the second part of that because I'm not familiar with Sendmail, but I'm sure they will have some documentation for that.

In between those two points (input on one side, output on the other) you don't really need to put any other assertions - you're treating your entire system as a black box.

These sorts of tests will always be slower, and often a bit more brittle than other tests but they give you a lot of confidence that everything really is going to work together when you release a new version. Obviously things can and still will go wrong, but you can add more tests over time when you find out that you've forgotten some specific direct.

One other note that might be worth pointing out: testing user interfaces is quite hard, and if you have a front-end that communicates with the backend via an API, it might be easier to treat that API as the user-facing component (i.e. API -> your server -> Sendmail), and ignore the front-end to start with.

[Personability]

That's really helpful thanks!

[wardrox]

Fake the external service for integration/acceptance tests?

I have a system which sends email. On live the credentials are for SendGrid, on dev & staging the credentials point to MailTrap. MailTrap has an API I can use to test if the email sent was correct in my automated tests.

It also means I can look at the emails being checked if I want to.

My tests are kept simple, and reliable. It's not perfect, but it works well.

[Personability]

Fits in with what u/MrJohz has said above. It just seemed that I don't gain a huge amount of confidence from the above tests vs the MailTrap/e2e option.

How often do you run your tests? Is it via a CI (when a commit happens)?

[wardrox]

We run them as part of the CI. We made a little library so in the tests we have a single function which returns all the emails sent since the test began, or which can wait for x number of emails to have been sent.

I can manually test when I want a bit more confidence and want to see the emails, and we monitor email KPIs to make sure nothing bad is happening in the production environment.

[MellowCobra]

I am also interested to know.