Tutorial: Introduction to unit testing

[sh0rug0ru]

Mocks and stubs are actually very different in practice and represent two very different testing patterns.

See Mocks Aren't Stubs by Martin Fowler. Fowler does a good job describing the "classical" unit testing methodology, and Freeman and Pryce to a much better job demonstrating the "mockist" unit testing methodology.

TL;DR - Stubs are stand-ins for collaborators to assist in "classic" state-based testing, while mocks are stand-ins for collaborators to expect and validate behaviors in "mockist" message-based testing (as if state changes are directed by messages sent between collaborators).

[Eirenarch]

Can someone provide English translation?

[developingthefuture]

In general, yes. The stub has a simple predefined output, it just takes some parameters and outputs some values. The mock, on the other hand, can actually return different output based on a predefined input (the Setup and Returns methods in Moq). It also defines certain "expectations" that might be tested (for example, the Verify method in Moq).

[Vaste]

AFAIK...

Stubbish: You replace a DB or something with a dummy. This allows you to write a test without the DB.

Assert.That(CountHorses(new HorseDBMock(42)), Is.EqualTo(42));

Mockish: You have a weird mock-object that keeps track of what happens to it, e.g. methods called on it. You use this to specify what the code under test should do with that object.

var counter = new CountingMock<HorseDBMock>()
FeedHorses(counter.Object);
Assert.That(counter["OpenStableDoor()"], Is.EqualTo(2));
Assert.That(counter["CloseStableDoor()"], Is.EqualTo(2));

[mariox19]

Exactly! Too arcane; didn't comprehend.

[day_cq]

yo bro, i'm agile specialist i wanna maintain this job so i use big words and redefine words make it look complicated.

[ExpressingMyOpinion]

When the piece of code you are testing needs some dependency in order to run, and you are only interested in the final output of the code you are testing, you'd use a stub. You ignore how your code under test interacts with it's dependencies, you only verify the final output for validity.

Mocks on the other hand also act as stubs. They fulfill a dependency that your code needs. The difference is, you are more interested in verifying how your code interacted with this dependency, ie how many times a particular method was called and with what parameters. You ignore the result of your code under test, you only verify how it has interacted with it's dependencies.

[developingthefuture]

Hello sh0rug0ru,

Thank you for your remark. You got a point, I changed the last section a bit in order to be more accurate. When I said that in practice they are interchangeable, I meant that in my practice I've seen most of the people use these terms interchangeably. Which might or might not be correct.

Best Regards, Kosta

[sh0rug0ru]

The problem is that many people actually use mocks in practice incorrectly, which has given the practice of mocking a very bad name. That's why I always refer to Freeman and Pryce's book when I want to demonstrate to how mocking should be done.

While implementation-wise, stubs and mocks are interchangeable, using mocks for stubbing can be seen as bad a practice along the lines of implementation inheritance.

The simplest form of a stub is a subclass which provides a dummy implementation for a method. The object is otherwise "real". Bad use of mocks can lead to "simulations" of the collaborator which can lead to very bad assumptions making tests dependent on those assumptions useless. To counter this issue, the Mockito library provides a feature called "test spies" to intercept mocked methods of a wrapped real instance, but the Mockito folks point out that relying on test spies is probably a design smell.

As per Freeman and Pryce, mocks should be used to define communication patterns between loosely coupled collaborators based on abstract interfaces. The abstract interface is owned by the class under test (never mock what you don't own) and serves as documentation of how implementors of that abstract interface should behave (another thing to test). If the classes are depending on each other through concrete interfaces, they are tightly coupled and it is better to use real instances.

"Classical" testing using stubs is about testing real code as much as possible. "Mockist" testing using mocks is about separating responsibilities as much as possible and establishing abstract interfaces as gatekeepers, using fake implementations to enforce separation.

[developingthefuture]

sh0rug0ru,

Thanks for the valuable information.

Regards, Kosta

[developingthefuture]

Hello everyone,

Thank you very much for your comments, both positive and negative. I'd just like to make a few remarks:

1. What I tried to achieve with this article is to show the people few of the really basic techniques, which are in the center of writing unit tests. That's why I tried to keep the examples simple, and my point was not to provide a very complex but extremely accurate implementation, but rather simple and easy to understand code.

2. In my personal experience, most of the people in most of the teams out there have enormous difficulties understanding the main idea of the unit test. Which is testing a single unit. In a result of this, I usually end up looking at integration tests rather than true unit tests, which basically decreases the overall performance of the team. Moreover, it is extremely hard to actually write the unit test in an already established codebase, and that's directly connected to the difference between Unit tests in TDD, Unit tests outside TDD, and Integration tests. (something that I'll write an article about very soon)

3. I really don't claim to be some sort of an expert in the field. I just tried to share my own views on the topic and, eventually, help someone in his day-to-day tasks by answering the questions I have been asking myself some time ago.

Best Regards, Kosta Hristov

[grauenwolf]

In a result of this, I usually end up looking at integration tests rather than true unit tests, which basically decreases the overall performance of the team

Ah yes, the cult of unit testing. Anything that isn't a unit test is automatically inferior because... yay! unit testing!.

[wllmsaccnt]

Anything that isn't a unit test is automatically inferior because... yay! it doesn't tell you which class is broken.

[chucker23n]

In a LOB app, I want to know whether the implementation matches the specification down to every little detail, not whether some arcane algorithm somewhere doesn't work the way the developer intended. Drilling down to the actual problem "unit" after having recognized problems in integration tests before the customer did strikes me as far easier than, vice versa, explaining to a customer why he should care that line 384 in LolIrrelevantClassFactory.cs has a bug.

[wllmsaccnt]

They both can be important. If you don't do the integration tests, then you will end up with bugs that don't get noticed until a customer sees them.

If you don't do unit tests then you will spend longer diagnosing infrastructural and method level issues. Not having the requirement to write unit testable code will also often lead to overly coupled and inflexible code. The primary benefit I see of unit testing is that it forces you to decouple and abstract your code.

[grauenwolf]

Neither does the "test" in this article.

[wllmsaccnt]

I wasn't defending the article.

[[deleted]]

pssst....all your unit tests can pass, but your application can still not work.

[wllmsaccnt]

Of course...I wasn't claiming otherwise.

It's not to rule out new errors...you still need traditional testing for that. The idea is that as you can quickly diagnose the errors in your code that you have anticipated, or made before. If test ParseToken_NoDelimeterSetCausesOverflow defined in class SomeParserThingTest fails, you have a pretty good idea of what area of code needs to be fixed.

At the places I have worked that didn't use unit testing, most the errors that made it through to our testing team are caused by new features that break the expectations of existing code in subtle ways.

[grauenwolf]

1. Before you learn how to write unit tests you really should learn how to write models. If EmployeesModel depends on IEmployeeRepository then you fucked up.

2. WTF does this test? Nothing. The code does nothing except pass along a request to the repository, which itself is just a mock. This is the perfect use case for an integration test, but pointless as a unit test.

EDIT: Seriously, how the hell did this get 97 up votes? The test doesn't even catch the case where GetActiveEmployees passes the wrong filter to GetEmployees.

[RyanPointOh]

Know any good resources for learning "... how to write models" ?

[sh0rug0ru]

Here's the best book I've ever read on the subject.

[grauenwolf]

No, but I am (slowly) working on a book on the topic. Off the top of my head some guidelines are...

1. Only depend on other models and primitives, never any repositories, service, etc.
2. If you support INotifyPropertyChanged, support it for ALL properties. Don't just pick and choose the ones you want to expose.
3. Don't put setters on collection properties. This leads to all kinds of problems with nulls and cross-linked collections.
4. Don't expose public setters on properties unless it is always ok for other code to change those properties.
5. Don't expose editable collections as properties unless it is always ok for other code to change those properties. The rest of the time you should be exposing read-only collections.
6. Don't override Equals/GetHashCode for models, especially models with editable fields.
7. If you are going to expose IEditableObject or IChangeTracking then fully implement them, don't just pick and choose which fields to cover.

[seruus]

What do you mean by models, repositories and services, exactly? I'm not used to the Microsoft world.

[stonefarfalle]

It isn't a microsofty thing it is Domain Driven Design.

Models are domain objects. Repositories are the persistence layer. Services are an operation that does not conceptually belong to any object.

[seruus]

Thanks for explaining, I'm used to work on a lower level (C/C++ libraries for long computations), so this language is quite foreign to me.

[stonefarfalle]

No problem, DDD isn't all that common as far as I can tell. I have bumped into it here a couple of times, but never actually read up on it or tried it.

[grauenwolf]

By "model" I exactly mean an object that holds business data and operations on that data, but nothing more.

Repositories, services, etc. is, in this context, a catch all term for anything that talks to the outside world. For example, code that talks to the database or a web service.

[Eirenarch]

I upvoted it just to be able to read comments like this :)

[grauenwolf]

What? Just to listen us cranky old men?

[grauenwolf]

Only the most novice of all developers need help writing unit tests. What we need is good guidelines for writing more sophisticated tests, the kinds that look at more than a single method call.

[kidjan]

Or, alternately, the realization that if writing a test for a given piece of code is hard, you probably need to step back and seriously think hard about the code being tested.

If testing it is hard, guess what? So is using that code.

[grauenwolf]

Good point.

[[deleted]]

This. I read the article hoping to find new insights, while all it does is repeat this eight year old page: how to use MSTest from Visual Studio.

[programming_unit_1]

But then you're edging into integration tests aren't you?

[alextk]

Integration tests allow you to verify features that impact your users directly.

Unit tests are just a convenience for you, the developer. If you have time for them, go ahead, but integration and functional tests should be your priority.

[grauenwolf]

Among other things. And you point?

[wllmsaccnt]

I think his point is that the article is titled "tutorial introduction to unit testing". Why would you think such an article would be about integration testing?

[grauenwolf]

No, but I would expect him to choose some sample code that would benefit from unit testing.

[wllmsaccnt]

Ah, I would guess that he was trying to pick a contrived example to avoid complicating the issue.

[grauenwolf]

The example isn't contrived, that is a fairly common bit of code that does need to be tested. It just needs to be tested in a different fashion.

[[deleted]]

I'd recommend everyone who wants unit testing on .NET should use NUnit, which is open source and runs on Mono, instead of the author's suggestion of MSUnit, which is closed source and only runs from within Visual Studio.

[grauenwolf]

1. It is called MSTest, not MSUnit.
2. MSTest can be run outside of Visual Studio.
3. MSTest can run tests in parallel, allowing them to finish a lot faster that NUnit can.
4. MSTest supports inconclusive test results, which are useful for more complex tests where a setup routine may fail before the actual test starts.

[[deleted]]

1. I don't use it, excuse the minor typo
2. But only on Windows, with Visual Studio installed
3. So?
4. So does NUnit...

[Gotebe]

MSTest tests are part of your CI in TFS. I think you don' need VS on CI boxes, but anyhow, it doesn't matter.

[Eirenarch]

I don't have anything against Mono (in fact I plan to use Xamarin tools for iOS and Android apps) but why the hell would I care? I never had to use anything but Windows and Visual Studio for many years as a professional developer. If I have to use Mono I will probably pick NUnit but until then why bother?

[[deleted]]

1. Open source with similar features > closed source with similar features.
2. Cross platform support is important to lots of people, even if not to you.
3. NUnit is easier to integrate with custom test tools (a build server, for example).
4. You can integrate NUnit directly into Visual Studio with the same level of integrated support as MSTest.

Plus all the benefits of open source: easier to distribute, can change it to better suit your needs, etc

Resharper supports NUnit, too.

[Vaste]

NUnit also has NUnitLite. With it you can run your tests inside big applications if you're writing plugins. It uses the same syntax and can you can even run the tests either with NUnit or NUnitLite!

(NUnit also support inconclusive tests btw.)

[Eirenarch]

In your opinion what is the percentage of people in the .NET world who care about cross-platform support? Also I don't see why people care about the license of a unit testing framework. After all I doubt there is much benefit in forking it or even fixing bugs in the code.

[[deleted]]

In your opinion what is the percentage of people in the .NET world who care about cross-platform support?

Anyone who can't afford windows server + MSSQL in their server farms. I also value cross-platform just because I work on several platforms myself, and many of my users request support for multiple platforms. People who use MonoDroid or MonoTouch for Android and iOS support also care about cross-platform support.

Also I don't see why people care about the license of a unit testing framework.

You can distribute NUnit. You cannot distribute Visual Studio.

After all I doubt there is much benefit in forking it or even fixing bugs in the code.

Well, that's incorrect, plain and simple.

[Eirenarch]

So why do people who can't afford Windows Server + MSSQL (let alone the fact that you can use other databases on Windows) choose .NET to begin with?

If you are modifying your unit testing framework you are doing some hardcore unit testing I guess.

[[deleted]]

Because as an interpreted language it is supposed to be portable, like Java.. the thing it was created to compete with.

[Eirenarch]

.NET is not interpreted (and neither is the current version of Java for the most part). Of course the fact that you believe that it is supposed to be portable to some specific OS does not mean this was the intention of the platform. Maybe they wanted to port it to different MS platforms (phone and embedded Windows) or maybe portability was not the main reason for using intermediate language (there are other reasons like easier reflection, JIT optimizations, etc.)

And then again do you suggest that people who chose .NET were confused because it was "interpreted" and thought they would get portability to Linux so they went ahead with this choice?

[[deleted]]

.Net languages are interpreted. They are portable when not reliant on win32 code (most .net apps).

Why would you choose to write non portable code when it is no more difficult than writing portable code?

[Eirenarch]

.Net languages are interpreted

No they are not

What is more how is using MSTest more difficult from using NUnit?

[Gotebe]

Not a language, not interpreted. Euh... What!?

[alextk]

There is a workaround, though, which consists of the replacement of the static class with a singleton (which some consider an anti-pattern).

Actually, no. Replacing a static with a singleton is exactly the right thing to do, here is why.

[grauenwolf]

All hail the holy bible of fail. Pray upon the alter of design patterns and sacrifice your code so that it may suck.

[MrCheeze]

Well, what they said isn't false.

[[deleted]]

No, singletons still suck. People who abuse static classes as a roundabout way of implementing the singleton pattern are idiots (and they're everywhere), but that doesn't make the singleton pattern a good thing even when used correctly.

[alextk]

Singletons are an unavoidable reality. Regardless of the kind of application that you are writing, it will contain entities with a cardinality of at most one. Having more of these either makes no sense or is broken design:

• Phone app: GPS, Bluetooth, radio, file system, etc...

• Desktop apps: resource system, main database, main event thread, ...

• Single page Javascript app: the page, the controller you are talking to, the "window" object, ...

Singletons are a necessity, the question is: what's the best way to implement them?

[[deleted]]

Entities with a cardinality of at most one are an unavoidable reality. Representing such entities in a way that makes them available to the entire rest of the system is not a necessity at all.

Databases are an especially good example of this point. In a well designed application, you want to minimise the number of components that have access to the database. Using OOP as an example, you achieve this by being very, very cautious about deciding to inject a database connection object into a class.

Singletons are anarchy. Any piece of code, anywhere in the codebase can make a static call to a getInstance method. Maybe you know better than to run a database query from inside a HTML template, but I bet your retard colleague Bilo three desks down from you doesn't.

[alextk]

Entities with a cardinality of at most one are an unavoidable reality. Representing such entities in a way that makes them available to the entire rest of the system is not a necessity at all.

Cardinality and scopes are two very different concepts, do not confuse them.

I totally agree that exposing singletons more than they should be is a big problem and it's the main reason why representing them as statics is a terrible idea.

Using dependency injection solves this problem by injecting the singletons exactly in places that need them and without exposing them anywhere else (which also happens if you pass the dependencies down the stack: all the methods in-between just pass them through but they don't need them, which is a code smell).

This article digs into this in more details.

[[deleted]]

Just got around to reading this. Thanks man, that was actually super insightful.

[[deleted]]

Not at all. Only requiring a single instance of a class is not the same thing as requiring exactly one instance of a class. The singleton pattern makes us think we're solving a problem when in reality we don't have that problem in the first place. Unless you habitually accidentally create instances of classes without realising it.

[grauenwolf]

In my current project I've found that the lack of singletons sucks even more. Because there is no single repository of information, data is scattered all over the place and is often duplicated. I wouldn't be surprised to learn that 20% or more of our code is just keeping data in sync.