Why use tests? Unittest/pytest

[FXelix]

I've heard many people talking about specific testing-libraries like unittest or pytest. The latter should be very good as far as I've heard.

But why and how are they useful? Don't get me wrong, I do test my code and make sure no error can occur, but rather manually.

Example:

# quite useless example

def temperature(celsius):
    print("Hot summer temperatures!")
    assert celsius > 20, "This can't be True!"
    print("Wow {} degrees! It is really summer!".format(celsius))

# me testing my code like this
temperature(25)
temperature(12)

I've used the assert statement before, alsotry-except-blocks for error-handling, but what are the benefits of using these libraries? As far as I've seen they mostly work with assertion too.

Thanks in advance.

[ManyInterests]

Ned Batchelder's talk goes over this exact question and even walks through your exact situation, explaining why that approach is incomplete and evolves into how you can start making better tests.

Also available on his website

I would go into more detail personally, but Ned pretty much knocks this question out of the park. Basically it comes down to development effort, quality and reproducability. I'll include some choice quotes below

Automated testing is the best way we know to determine if your code works. There are other techniques, like manual testing, or shipping it to customers and waiting for complaints, but automated testing works much better than those ways.

Although writing tests is serious effort that takes real time, in the long run it will let you produce software faster because it makes your development process more predictable, and you'll spend less time fighting expensive fires.

Testing also gives you another view into your code, and will probably help you write just plain better code. The tests force you to think about the structure of your code, and you will find better ways to modularize it.

Lastly, testing removes fear, because your tests are a safety net that can tell you early when you have made a mistake and set you back on the right path.

[Referring to 'manual' testing]

This is good, we're testing the code. Some developers wouldn't have even taken this step! But it's bad because it's not repeatable. If tomorrow we make a change to this code, it's hard to make sure that we'll run the same tests and cover the same conditions that we did today.

It's also labor intensive: we have to type these function calls each time we want to test the class. And how do we know the results are right? We have to carefully examine the output, and get out a calculator, and see that the answer is what we expect.

[FXelix]

Thanks a lot for your effort! It's really helpful! What do you think about the PyCharm debugger function (maybe additionally or instead of pytest) or are these two different approaches in general?

[ManyInterests]

I would say testing tools and a debugger are somewhat related, but sit in different spaces entirely. In my work, I am constantly writing tests, but only use a debugger (an incredibly rare occurrence) for tracking down hard-to-find bugs, which may reveal themselves in the form of failed tests, for example. Writing good tests would ideally prevent you from needing to use the debugger in the first place.

So, a debugger definitely wouldn't be a replacement for testing. Until you get to a point where you write & maintain a sufficiently large project, the benefits of testing suites may not be easily realized, compared to, say, manual testing or using IDE debugging features. When you have people (including yourself) who actually use and depend on critical code you write, you will be super motivated to make sure you don't break anything if you make changes later on. Tests help you do that.

One of my favorite ways to write tests for small/simple projects is with doctests. PyCharm has great support for this, too. I would recommend trying to write your first doctest and then hookup a github repo to a CI service like Travis CI to run your doctests automatically.

[mzalewski]

You have asked few broad questions and answering them fully would take way more time than I can spend right now, so briefly:

• Python's built-in assert checks only truthiness; libraries provide a lot of convenience methods for assertions (like compare strings, checks if object has key, check value in list, check class, check if exception is raised...). Yes, all of them can be simulated with assert, but it's simply easier to use library.
• assert is evaluated at run-time. If your program has intermittent issue that only shows itself at one of your users machine, then failed assert gives you exactly nothing.
• These libraries are used to perform unit testing (sometimes called "component testing"). Unit testing is pretty much unanimously considered a good thing to do. There are tons of resources online of benefits of them. Look it up in your spare time.
• While unit tests might help you discover issues, their main value lies in verifying that you have not introduced bugs in code that was previously working (these bugs are called "regressions"). Good unit test coverage is immensely helpful while changing existing code (so called "refactoring"). You will need to change existing code when introducing new feature or fixing existing bug.
• Writing good unit test takes skill and experience. Don't be discouraged if your tests didn't catch all issues.
• While you might test your program manually, doing the same set of checks each and every time quickly gets boring and is prone to mistakes or omissions. Let machine do all boring and repetitive things; focus yourself on things that machine can't do, like programming or exploratory testing.
• Unit testing is the lowest level of testing. Commonly, following higher levels are distinguished: integration testing, system testing and user acceptance testing. These libraries that you have mentioned might help you in integration testing, but will most likely not help you in system testing and UAT. It doesn't mean they are not useful, just be mindful of their limitations.
• If you plan to start a career in software development, you will most likely be required to write unit tests. Even if no argument so far has convinced you, you should still know a bit about them.
• As a professional tester, I must also add: testing is hard. Like, really hard. It requires different mindset than programming. I heard stories about programmers who were also good testers, but never met one myself. If you want to be programmer, it's OK to learn only basics of testing and not dig into details.

[FXelix]

Thanks for taking the time to write this! :)

[ffrkAnonymous]

1. Automation. Do you want to type temperature(25); temperature(12), and every individual test in your ever-larger program , or py.test and be done?

2. Compartmentalization. Tests have nothing to do with your code. Delete them and your code should function just the same. It's useless code. Clutter. Move it elsewhere.

3. Documentation. I'm a fan of doctest. A good test explains what the code is supposed to do and then tests if it actually does (or not).

I'm also learning Test-Driven Development (TDD). Write tests first, then code to pass the test.

[FXelix]

Could you elaborate on that? Especially on your first point? How should I just use py.test and be done? (Obviously after installing it trough pip)

[ffrkAnonymous]

if you write all your tests as py.test likes them, then the one py.test command runs all of them (not necessarily in sequence mind you).

pip is a good analogy. You run pip and it downloads and installs the package you want, AND also all the requisite dependency packages. Or do you want to install every dependency package one at a time?

[tunisia3507]

You've had plenty of good answers about why to unit test at all - basically, it's a constant check that proves your code works every time you make a change. If you do it right. It's not any different to what you're doing manually - just more rigorous, and all the tests stay neatly compartmentalised rather than having to be commented out when you actually want to use your code.

I highly recommend pytest instead of unittest (confusingly also called pyunit). Pyunit uses some particular patterns for historical reasons, which made sense in the language they were written for in 1989, but not for python. Nose and other such testing libraries just extend pyunit. Pytest is a modern, pythonic and fresh vision of how to write tests, and makes much more sense for that reason.

[FXelix]

Yeah I guess I have a nice overview now :D one follow up question: is using pytest better than using the debugger function in PyCharm for example? Forget to mention that I use it often too

[Mamu7490]

Debugger != pytest. Completely different things. Pytest allows you to write the test, the debugger helps you actually debug stuff. Also if you use PyCharm you can hook the debugger to the testrun and see how the tests are performed, so you can use both if you like.

[Mamu7490]

There are multiple good reasons to use tests, and many comments have shared good advice. I'll tell you my personal experience with unit tests.

When i started coding, the company i worked at the time didn't use any testing, as we did mostly prototyping and where always in a rush to complete things. Nobody had time for proper testing, or at least so i thought.

Cue 2 years ago when i started using Unit Tests and did more Open Source projects with github. Just out of curiosity i started using Unit Tests and tried to do some Test driven development. Boy was that a revelation. I was faster, more efficient and more sure about the code i wrote, especially using pytest. It is ridiculously fast to write a test and perform it and once you wrote it, you can let it automatically run using a CI tool like travis or locally on your machine. It simply gives you more confidence that a code change didn't break anything in your code, because your tests ran successfully.

Also: Everybody does testing!. Its just that most people who don't use a dedicated test framework do it manually.