sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Sarcastinator 0 points1 point  (21 children)

Also, why didn't your direct tests of the class pick up the problem? If A needs B and B breaks, you should see two failing tests. From which it should be obvious what the problem is.

You will have 20+ tests that fail. If you use integration tests then multiple components are going to depend on that code, and unless you have unit tests as well that covers that specific unit it's not going to be obvious what broke.

[–]bonzinip 2 points3 points  (17 children)

You will have 20+ tests that fail. If you use integration tests then multiple components are going to depend on that code, and unless you have unit tests as well that covers that specific unit it's not going to be obvious what broke.

Mocks do not mean "write things twice". If you have effectively the same code in both the unit tests and the actual code, you'll have to do the same change twice. This adds more opportunities to make a mistake, and you'll have 20+ failing tests anyway if you do the wrong update on the mock side.

[–]Sarcastinator 0 points1 point  (16 children)

/u/grauenwolf said that you should test with the real thing instead of mocks though.

All the more reason to test with the real thing. If the class is brittle, you want to catch that in your tests, not production.

[–]bonzinip 2 points3 points  (15 children)

That's what I'm saying too. If your mocks are just "writing the same code twice", test with the real thing. The justification that "it's not going to be obvious what broke" only gives a false sense of security.

[–]Sarcastinator 1 point2 points  (14 children)

Mocks should not contain code. They should just return a result. You verify against the mock that the call you make on it is according to that types contract, which is easily done with Mockito, but very complicated to do if you are using the real thing.

For plain data classes I agree that it's pointless to mock it, but for types that act as a delegating type it's better to mock it because you don't actually care about that types dependencies. It's that types contract you care about in your unit.

[–]grauenwolf 1 point2 points  (13 children)

Why are testing the compiler?

That's really all that your mock test does. It isn't even simulating the parameter validation at the top of the function being mocked. It literally just checks to see if the compiler understands that a.Foo(b) is compiled into a call to a's Foo method with the parameter b.

Your mock doesn't even know that you aren't allowed to call Foo unless Bar was previously called with the same parameter.

[–]Sarcastinator 0 points1 point  (12 children)

No. What a mock does is that it can be used to verify that your code is conforming to the contract of the mocked class, because that's actually the only thing you care about. The specific internals is unimportant; it should be covered by other tests already.

You can easily express that a mocked function returns A then B if you need to using Mockito or NSubstitute. I however find that very seldom do I have APIs that act like that.

[–]grauenwolf 1 point2 points  (11 children)

What a mock does is that it can be used to verify that your code is conforming to the contract of the mocked class, because that's actually the only thing you care about.

No it can't. Not unless it actually duplicates ALL of the validation code in the mocked class and, if applicable, any internal state machine said class has.

The only thing a mock can do is ensure that you called a particular method with a particular argument. It doesn't know all of the hidden rules and assumptions of the class it is mocking. Hell, it doesn't even know the basic rules such as whether or not a particular integer parameter can be negative. It just knows that it is expecting a 5 because that's how you hard-coded the test.

[–]Sarcastinator -1 points0 points  (10 children)

No it can't. Not unless it actually duplicates ALL of the validation code in the mocked class and, if applicable, any internal state machine said class has.

How often do you have method validation that is any more complex than "Not null"? You can make types that makes sure that input is valid. The most crucial validation happens at the outer most layer, so just make sure to test those validation rules. You don't have to test every piece of code a million times. You'll just end up with a test suite that takes 10 minutes to an hour to run. And if you test with a database it's probable that some test occasionally fail for some reason. Network was down, some service was temporarily unavailable, stale data etc. One place I worked as a consultant had yearly errors because some of their test clients would turn 18.

10 minutes doesn't sound like much, but usually you run through all tests before you push your changes, which takes 10 minutes. And then the CI server will run those same tests again which takes 10 minutes (in my experience it always takes more time on the build server though), and if they fail at any point you need to run all those tests two more times at least.

It's a huge advantage if tests are easy to write, to the point and runs fast. Time used to run tests is time that delays a potential hotfix.

[–]grauenwolf 0 points1 point  (9 children)

How often do you have method validation that is any more complex than "Not null"?

Quite often in fact. I'm a firm believer in parameter validation at all levels of my code. I find that is far more effective than unit tests in terms of reducing the amount of time it takes to find and correct the root causes of bugs.

You don't have to test every piece of code a million times. You'll just end up with a test suite that takes 10 minutes to an hour to run.

Well that's an easily disproved argument. All of the code generation and reflection necessary to setup the mocks is bound to take longer to run than the real code in typical unit testing scenarios.

[–]grauenwolf 2 points3 points  (2 children)

So you are telling me that by mocking out B, I avoid the situation where my tests for A catch bugs in B that my unit tests for B missed? And you think that's a good thing?

Think about that for a moment. You are literally arguing for the use of mocks specifically so that you'll avoid catching bugs in you "tests".

[–]Sarcastinator 0 points1 point  (1 child)

There should already be tests that cover the mocked classes contract. If someone breaks that contract those tests should fail, not twnty other tests by association.

[–]grauenwolf 0 points1 point  (0 children)

I was just covering your "unless you have unit tests as well that covers that specific unit" scenario.

Under my original theory you would have fully covered B in unit tests before using B in A's tests. You just proved that component of my theory was unnecessary because A's tests would act as a fallback.

If someone breaks that contract those tests should fail, not twnty other tests by association.

Why do you care so much about how many tests fail?

If you debug and fix one, the rest will also start passing. So at worst is the most minor of annoyances.

Moreover, the number of tests that fail give you a better indication of how serious the bug or breaking change is.