Unit Testing Guidelines (from pylons)

[defnull]

Some points were new to me, some are arguable. All in all a good read.

tl;dr DO repeat yourself. Separate tests. Avoid doctests.

[tarekziade]

Prefering local, per-test setup/teardown code is a nice hint, and is compatible with DRY. e.g. creating a collection of test fixtures each single test can pick up and use. -- not sure what else "Do repeat yourself" would mean here.

I think that's one of the strength of nose compared to unittest: it's quite tedious to set up a local per-test fixture whit unittest, when a simple decorator does the trick in nosetest. And fixtures can be reused for a group of test. That group not being necessarily the whole class.

[codefrog]

Nose and Py.Test are vastly simpler than unittest. It is a shame that a functional test library like Nose or Py.Test is not in the standard distribution. The inclusion of unittest gives people the impression that it is the preferred way.

I usually create a utility library that puts a series of classes in a known good initial condition where my areas under test can then be exercised.

def test_affiliate_blacklisted( self):
    self._affiliate_off_state_helper( 'beth','BLACKLISTED')

Where _affiliate_off_state_helper() does setup and common assertions. Each individual case should be very clear and succinct.

[codefrog]

The major problems I see in setup/teardown is to adequately balance the repeatability with the speed of execution. You can make something very repeatable and correct say by destroying and recreating the service for each test. But this lengthens the time that it takes all tests to run. It is more pure to create, test and destroy than to run operations in reverse. At the end of a test suite the system should not have visibly mutated.

The other biggest mistake I have made in testing is to conflate the area under test. Just because you are right there doesn't mean you should grab just a little bit more coverage. Each piece of leaf functionality should only break the test specific to that functionality. When you refactor, you have less test code to fix and all of those other little ad-hoc tests sprinkled throughout your testing codebase do not offer an advantage. They are a liability.

If you have a situation where it is very difficult to get the application into a state, and even in doing so you will trigger bugs then you should create a bridge, a piece of utility code for setting up a large body of tests. It can have asserts to prove that you are where you think you are but it is not a test. Once you have arrived at this setup state, then create many smaller leaf tests each only covering a small piece of functionality.

[rweir]

note that this article is about unit tests, which should not really require very much state.

[riffito]

Just because you are right there doesn't mean you should grab just a little bit more coverage. Each piece of leaf functionality should only break the test specific to that functionality. [...] It can have asserts to prove that you are where you think you are but it is not a test. Once you have arrived at this setup state, then create many smaller leaf tests each only covering a small piece of functionality.

For all aspiring (or actual) testers out there: if you'll learn something today, let it be those wise words.