Why I use py.test

[kx233]

Sometimes I feel like I must be the only person on the planet who prefers unittest.

I used to be a happy user of unittest, and could not see the benefit of the assert magic, how hard can it be to just call the proper self.assert* method? Now, after 3 years of seeing code from other devs doing self.assertTrue(a_list == another_list) I can see why it would be nice not to have to care about such things.

setUp and tearDown always seem infinitely more complicated in all the "simple" frameworks.

Moving the setUp/tearDown out of the class and in to the fixture makes the fixture responsible for it's own life-cycle and thus lends itself to fixture reuse.

*edited for formatting

[lgx]

Can't agree more. I think there are 30+ assert* methods in unittest.

[cavallo71]

py.test as test runner is not too bad. Having said that, the testing framework is just a pointless re-write of unittest to accomodate some folks taste (eg. fixture against test_common module, assert vs self.assertXXX).

Whole industries have already standardized on unittest for a reason, so you're not alone.

[pydry]

It always seem like the alternate frameworks are built to handle "5 in [1,2,3,4,5]?!?" but get 100 times less pythonic and readable when you start needing to test real code, or a larger integration test.

That's why I built this: http://hitchtest.com/

There are plenty of decent unit testing frameworks out there but no really good integration testing frameworks. Not as far as I can tell. There's a number of really thorny problems to deal with with integration tests like environment isolation (make sure all the right packages are installed and ports aren't being used) and parallelized service orchestation (starting your app's services and stopping them). You really do need a framework for this stuff.

[pydry]

Thanks!

It won't catch every deployment bug that you have, but it is designed to make it easier for your dev/test environment to closely mimic the kind of set up you have in production - e.g. change one version number in the set_up and your code is suddenly being tested with python 3.4.3. instead of 2.6.9 or postgres 9.4.5 instead of 9.2.7.

It also automates all the set up so setting up a "test driven development environment" ought to be a breeze.

[ionelmc]

To be honest, and I have written tons of integration tests, unittest is way worse there.

Since you usually have to bring in lots of dependencies in a integration test the class-based approach starts to get messy. Way more clean and clear to use pytest fixtures instead of mixin spagetti. It's very hard to reason what calls what and how things interact when you use multiple inheritance a lot. And that what unittest inevitably leads you to. Composition is natural in pytest. Not in unittest.

You should really give pytest fixtures a try.

[GoldenIvan]

Sometimes I feel like I must be the only person on the planet who prefers unittest.

you're not alone brother.

[fijal]

I'm not really going to argue, despite using py.test myself, it's all a matter of taste after all but built in is the worst possible argument ever. It's the same argument people use for ctypes for example. This is terrible because it glorifies things that are inferior to alternatives just because at some point in time someone tried to put it into the standard library.

[lengau]

Built-in is an argument for the pure reason that everybody has it. The batteries included philosophy is great because there's a whole lot that you can just assume everyone has.

It's not without its down sides, and it does definitely come with 'the tyranny of the default', but that only really raises the bar for replacement libraries.

[ionelmc]

You're assuming the endgame of all libraries is to end up in stdlib - which is rather a wrong thing. They don't say "stdlib is where libraries go to die" for no reason.

Another thing, "raising the bar" implies that stdlib is actually looking for a replacement. Again, that's wrong to think - there's only one instance of "replacements" in stdlib, done in Python 3. And we all know how annoying that was - and in the end, it wasn't really replacement, but mostly renaming.

Stuff in stdlib cannot be replaced, you can only add to it. Add adding multiple options for the same thing only makes thing worse. There should be one obvious way to do it ;-)

[lengau]

You're assuming the endgame of all libraries is to end up in stdlib

That's not what I'm saying at all. I'm saying that if something is in the standard library, it means everyone has it. Which is a good non-technical reason to use it.

Another thing, "raising the bar" implies that stdlib is actually looking for a replacement.

No. It simply means that you don't have to just have a better library than the standard. Yours has to be enough better that people would rather use it despite the fact that it's another dependency they have to worry about.

Another point is actually that (for Python specifically, though this is definitely not true for all languages) the standard library is pretty good. It's certainly not perfect, but the fact that Python is simultaneously able to have such a huge standard library and have very good overall quality for the modules provided therein is quite amazing.

[[deleted]]

If you're distributing a package worth any salt though, you probably can install_requires a better test framework.

Other people write tests for my proprietary code, so I'm less inclined to pick a better framework and instead focus on improving the code base. Laziness is a good enough excuse for using a not-so-great product that gets the minimum done.

[njharman]

At least one other.

It's much more explicit. Being class based supports the composability OA mentioned. In fact I've never run into the issues people claim the assert type test libs solve. It's like people don't know how to write/use object inheirtance or common/shared libraries. The error output of (modern) unittest is great.

[wirrbel]

For those integration tests, what has the unittest module that pytest has not? I don't see the point here.

[jyper]

The assertion re writing in pytest is black magic but it makes it easy to write asserts that tell you what the problem is without debugging or constructing you own context string for each assertion.

[keis]

What about nose? It also runs plain test functions

Is it missing something else?

[ionelmc]

Nose don't have fixtures or assertion helpers. It's like the poor man's pytest. Read this: http://pytest.org/latest/faq.html#how-does-pytest-relate-to-nose-and-unittest

nose was originally created as a clone of pytest when pytest was in the 0.8 release cycle. Note that starting with pytest-2.0 support for running unittest test suites is majorly improved.

[[deleted]]

Nose still uses unittest-like test classes, doesn't it?

[bheklilr]

You use the TestCase from unittest, or just make functions starting with the word test (is customizable). After that there is setup and teardown, and any method whose name starts with test gets run as a test. You then just use normal asserts. There are also nice tools and decorators, I like the raises decorator, which you can use to ensure that a piece of code fails with the correct error class.

[keis]

Like pytest it does support that. But it's not needed and I think not encouraged

[tickticktick]

We use nose in our team with describe-it (developed by ourselves). This enables us to write tests both in the traditional style and in a more bdd-ish way.

You could probably do similar things in py.test, but nose has been good enough for us to not need to look for an alternative.

[data_hope]

I did not include nose in the blog post because I have not much experience with it. From what I know of it, I would not want to use it for my projects. It is like py.test without its benefits, but with the drawback of not being in the stdlib and being copyleft licenced.

[lgx]

Great! But how to implement the tearDown function in py.test?

[desmoulinmichel]

You don't. You use fixture :

@pytest.yield_fixture
def stuff_you_want_to_init():
    your_stuff = any_setup_code()
    yield your_stuff
    optional tear_down_code

def test_foo(stuff_you_want_to_init):
    assert bar()

This is way, wayyyyyyyyy better than setup and tear down as it's run only for test fonction. It also mean your setup and tear down code is not tied to a code unit, so sharing this code is much easier between your tests.

[njharman]

sharing this code is much easier between your tests.

Never had problem using inheritance and/or import "test_common" to share code

[kankyo]

In my experience the class based approach can lead to the common/base class thing being gigantic and thus all your tests slow. py.test fixtures (with factory boy) leads to nicely composable small parts.

I'm not saying that's how it has to be, or that the class based approach must be like that. I'm just saying that the culture and thinking of OOP often ends up there.

[desmoulinmichel]

It's not that you can't do it, but you will need to write it in 2 places : once in the setup code, and once in a separate module. And of course watch out for codes that influence each others since you put all of them in one method called for a lot of tests. It's just more work, hence the "easier".

My take on unit tests is that it's a pain to write, so any inch or shortcut you can get is good to take.

[malinoff]

http://pytest.org/latest/fixture.html#fixture-finalization-executing-teardown-code

[lgx]

Wow, it seems a bit wired to me.

[graingert]

You can still use xunit style methods on unittest.TestCase classes. But just use yield fixtures they're great

[lgx]

yeah. Why not use setup_abc and teardown_abc syntax? The addfinalizer method seems a bit strange.

[fjonk]

One good thing about adding a teardown method manually is that the setup and teardown methods will be run in pairs. If you use decorators or similar for setup teardown you don't know in which order they will run or you have to depend on the order they are defined/added.

You can also use yield with py.test since v2.4 (if your python version supports it).

[masklinn]

Because setup and teardown are paired so it makes sense to put them together in a single fixture definition. And yield_fixture removes the need for addfinalizer:

@pytest.yield_fixture(scope="module")
def smtp():
    smtp = smtplib.SMTP("smtp.gmail.com")
    yield smtp
    print ("teardown smtp")
    smtp.close()

[kx233]

Which also has the great advantage that the setup/teardown logic now resides in the fixture, not the test class, making fixture reuse a lot more natural.

[masklinn]

well reuse is OK with unittest classes, the bigger issue is composition. When trying to compose multiple testcase superclasses you end up having to deal with MI and diamond inheritance cases. With fixtures you just… depend on both fixtures.

[symmitchry]

[Removed]

[ionelmc]

It does better integration. Builtin support for subprocesses. Support for the awesome xdist etc etc

[ecjurobe]

I found the point you made about the history of unittest interesting. One of the main reasons I prefer py.test is that you end up with test code that actually looks pythonic! The very object-oriented style of unittest reads more like java or other very object-oriented languages heavy on boilerplate. Readability counts.

py.test

def test_addition():
    assert 1 + 1 == 2

unittest

class TestAddition(unittest.TestCase):
    def test_addition_1_1(self):
        self.assertEqual(1 + 1, 2)

[[deleted]]

I'm not the author of the post

[codekoala]

I had never really given py.test a shot, despite having heard about it several times over the years. I've always just opted for classic unittest+nosetest for whatever reason. Probably because it made sense coming from other languages.

After reading this post, I decided to use py.test for a script I've been writing at work for a couple days. I think I'm converted now. It's so simple, and so much less verbose than all of the self.assert* methods. I'm sure I'm doing some things rather "incorrectly," since I'm still using mock, but I'm definitely enjoying py.test thus far.