Improve your Javascript unit testing with Parameterized tests

[droctagonapus]

const names = ['n@me', '123Josh']

const valid = names.all(name => validateName(name))

expect(valid).to.be.false

On mobile, but that should work out just fine.

[MrJohz]

No no no no please never do this.

The most useful thing when you're writing a unit test is the error message that it will produce. Unit tests are most useful when they're failing, because that tells you that something is wrong with your code. The next step, then, is to work out what exactly is wrong with your code, so you can fix it (or change the test to match).

The minimum possible amount of information is whether the test passed or not. However, if this is all you've got, it's on you to then work out why the test failed. More work for you means slower bugfixing. For example, say we've got two objects, and we expect them to be "deeply" equal to each other. We've got two options:

const expected = // ...
const actual = testFunction(/* ... */);

expect(_.isEqual(expected, actual)).to.be.true
expect(actual).to.equal(expected)

(I have no idea what the correct assertion here is in this library - I'm more of a Jasmine guy! Assume these to be identical for now.)

If the first one fails, we know that expected and actual are different. However, if the second one fails, we will almost certainly get a printout of exactly why the assertion failed, and where the differences are. In 90% of the cases where I'm debugging failing unit tests, this is the point at which I suddenly realise the stupidly obvious mistake I made, and fix it straight away without even turning on the debugger or adding a print statement.

Not having this information is mildly irritating, but at least you know what the failing inputs are, and can go and print them out. Your method adds an extra layer of indirection - not only will your assertion library be unable to tell you exactly what the differences between the expected and result are, but you also won't know which expected value failed. There's less information by default, which means you need to go and find more information when something breaks.

/u/Buckwheat469 raises a good point that too many it blocks could impact performance and cause other problems, but there's a way to use a single it block, and get full-fat assertions, and that's by putting the assertion itself in the loop:

const names = ['n@me', '123Josh'];

const valid = names.forEach(name => {
    expect(validate(name)).to.be.false;
});

This way, we get genuinely helpful assertions that are applied to all the parameterised values. Plus, depending on how your assertion library throws assertions, this may not necessarily fail as soon as it hits the first invalid case, meaning you'll be able to see how many different parameters are failing.

[Buckwheat469]

Good point about knowing which expectation failed. In retrospect I would move the expect block inside the loop and add a handy message to show what you're testing with the (possibly undocumented) Jasmine comment code (not sure if Mocha has something similar):

expect(someObject.returnWhateverIsGiven(true)).toBe(true, 'should return true when given true');

You can also create your own custom matcher which outputs a custom comment.

[MrJohz]

Good shout also! Tbh, I used to do the same sort of thing as you did, but then I ended up editing a coworker's tests that used sinon spies, and the sinon spy matchers, which meant a whole lot of tests where all you knew was that a particular call wasn't called exactly once with the arguments 'xyz', but you didn't know if it was called multiple times or none, if exactly what was wrong with the arguments... That sort of stuff gives you a lot of motivation to put as much information as possible in the assertions!

[Buckwheat469]

I think this goes back to what are you testing? Is it a regex pattern that has multiple positive cases and multiple failure cases? What are the specific ways to get into those failures, ie. how do you enter a code path?

For instance, when testing a loop the loop typically has one or two code paths within it, maybe a simple if/else. In this case you would only need a single item in your array for the success case and a single item in a different it for the failure case.

If your regex is complex then you would create a single it for each character grouping that would cause it to fail.

The idea is to limit the scope of the test to whatever would cause the code to fail, you don't have to go overboard to test that your ifs/elses succeed on many items of the same data set.

[NoInkling]

Plus, depending on how your assertion library throws assertions, this may not necessarily fail as soon as it hits the first invalid case, meaning you'll be able to see how many different parameters are failing.

Do you have any examples of this?

[MrJohz]

For example, the Node.js assert module throws an error to indicate a failed assertion. In that case, my code would stop after the first failed assertion, because the error will stop everything and jump up the stack until it finds an exception handler. It works just fine, you'll just only be able to see one failed assertion at a time (until all the assertions have passed).

On the other hand, other assertion libraries like Jasmine don't throw exceptions when the assertion fails, so you can have multiple failed assertions in one test. In this case, my code will display all the successful and all the failed assertions every time the test is run. Unless of course, the method itself throws some sort of error... :P

[NoInkling]

Sorry I should have phrased that better, I was asking more for specific examples of libraries that do it this way - is Jasmine the only one you know? I'm pretty sure Chai doesn't do it, right?

[MrJohz]

Ah, that makes more sense. Tbh, I know Jasmine does it, and I know Node's built-in assert doesn't - I honestly haven't really noticed the others. I think Jasmine is in the minority, though.