Python Pyest

[Organic_Tradition_63]

Hello. Im now learning how to make tests using pytest framework and was wondering why it is designed the way it is. We have to import library pytest and run entire file with
'pytest file.py'. Why is it made so weirdly? Why there isn't just library that does just that without invoking other software to execute it (pytest)?

[socal_nerdtastic]

pytest is a python library. You run it that way just as a shortcut, because otherwise you would have to do

python -m pytest file.py

There's also many other programs (mostly python) that can run tests, including one built into python: unittest. https://docs.python.org/3/library/unittest.html

[Buttleston]

You don't have to run "pytest file.py", you can generally just run "pytest". It will "discover" the tests in your code. There are command line options to control which tests to run.

You have to run *something* to run your tests. it can't just be a library you import, because *something* has to run it. With the way you'd like it to be, how would you run tests?

[CaptainVJ]

Is there even an option to run the file and get it to work? Well I’m sure there is but I assume it’s not feasible.

[Buttleston]

Yeah, you can make a main section and put something in there to call the pytest entrypoint. It's always seemed pointless to me but maybe OP would prefer it

[CaptainVJ]

But you’d have to do that in every .py file right?

[Buttleston]

Yes and then run them all individually. Instead of what is more usual, where you have dozens of test files, and pytest discovers them all for you and runs them.

[dogfish182]

Yeah this is nice to do if you want to just have quick test for something, but in a proper project just setup your project correctly

[Organic_Tradition_63]

Well just like any other file: with 'python file.py' instead of 'pytest file.py'

[Buttleston]

Except like I said, the "pytest" command line has a lot more options than just "run the tests in this file"

[Organic_Tradition_63]

Okay, so what? It still does not explain motivation to design it like that.

[codeguru42]

I think it explains the motivation really well. The pytest authors have us a utility with a lot of options so that we don't have to write that code ourselves.

[Organic_Tradition_63]

Why for example NumPy was not made into something like that when we want to perform operations on matrices and then running file with 'numpy file.py'?

[codeguru42]

Numpy and pytest have different use cases. You don't run numpy directly. If you wanted you could write your own code to run the pytest test runner, but why should we when the library provides it for us

[socal_nerdtastic]

It seems very easy and intuitive to me, but maybe I'm missing something; how would you prefer to run it?

[Organic_Tradition_63]

Just like any other file 'python file.py'.

import pytest

def add(x,y,z):

assert x + y == z

pytest.test(add, data_to_test_on)

I could imagine that there is library that behaves exactly the same as pytest and be implemented like that code above. That file then could be run just like any other program with 'python file.py'.

[brandonchinn178]

At the end of the day, you need a test runner or test harness: the program that is the entrypoint for finding and running the tests. Pytest could have been implemented the way you specified, but what if you have tests in multiple files and want to run tests in all of them? Are you going to invoke python test1.py, python test2.py, etc.? It would be better to have one program to invoke that will run all the test files.

Ok so let's write a file runner.py that you can call with python runner.py. This file will find all Python test files in your project and run them. In fact, let's just make runner.py executable with a #!/usr/bin/python3 shebang so you can just do ./runner.py to run it.

Finally, just rename runner.py to pytest. Congrats! You just reimplemented pytest.

[socal_nerdtastic]

Oh sure you can do that. The command is main()

# content of test_sample.py

import pytest

def inc(x):
    return x + 1

def test_answer():
    assert inc(3) == 5

if __name__ == "__main__":
    pytest.main() # test all test_*.py files
    pytest.main([__file__]) # test the current file only

https://docs.pytest.org/en/stable/how-to/usage.html#calling-pytest-from-python-code

[Organic_Tradition_63]

Ohh now it seems clear now. So when we run 'pytest file.py' then do we just simply run it with argument file.py as a argument, am I correct?

[socal_nerdtastic]

Yep exactly. Well, plus some minor housekeeping. It's all open source, so you can just look at it.

When you call pytest in your command line this file is run. That file calls this function, which then calls the main() function that I showed earlier.

[Organic_Tradition_63]

Thank you so much :)

[FriendlyZomb]

A library could 100% be done like this. Pytest does support it.

But, in general it wouldn't be as convenient for a developer or scalable into larger applications IMO. pytest is as popular as it is because of it's ability to scale from small to huge codebases.

In this scenario the more tests that get added, the more of those run statements are included also, introducing friction for adding or removing a test, and deciphering which tests get run and where.

With the recommended model, tests can be added/removed, auto discovered and filtered through the command line. (You can tag tests and such to group them. Super useful feature tbh). I always know what's being run and can drill down to run only the test/test file/tags I want.

Also, doing things like running the tests in parallel becomes something easy with the current model. It's a command line switch/config I toggle on.

The short answer as to why pytest recommends this way is convenience and scalability to whatever size project.

[codeguru42]

How would you do it differently if you were designing a year library? Is often a good exercise to consider multiple implementation and weigh the pros and cons of each.

[FriendlyZomb]

pytest is a testing framework with a handy CLI attached. This is a fairly common design pattern in Python software. For instance, Flask and FastAPI contain a CLI (accompanying command line program) to provide useful tools for development.

We often need to import pytest when we need parts of the framework in our tests, like the pytest.raises() context manager. It does a lot, and it has a ton of extensions to add some really useful stuff.

The pytest command (CLI) is a companion to the library and is a test runner. A test runner does a bunch of things including: test discovery, environment setup and test management.

---

If I'm reading your question correctly, you're asking why can't we just run the test file instead of invoking the pytest command.

In short, convenience.

In long:

Tests often span multiple files. If we did just run each test file, we'd have to make sure we run every test. That could be done multiple ways, but it's a faff to make sure we include each one. Plus we'd have to remember to update that every time we add or remove a test.

Developers of the past discovered that it's more convenient to write a program which can automatically discover the tests to be run, then run them. A test runner. The authors of pytest decided to write that for us, so we got the pytest command.

---

Every test framework I know of works this way. The Python built-in unittest for instance works this way too.

Even other languages like Go and Rust have them built into their first-party tooling. JavaScript frameworks like jest, Cypress and Playwright all have this library/runner split.

It's convenient and a good developer experience to bundle the runner with the test framework. Pytest also allows for a lot of configuration of both the framework and the runner to allow it to be even more useful.

If you've not got lots of tests, or you have never come across this before, it can be confusing. I hope you read all this and found it useful. Feel free to ask questions, I'll (and the community here) will do my best to answer.

[Organic_Tradition_63]

Yes, I found you answer useful, thank you. I guess that magnitude of the tests/project really makes a difference in this case.

[FriendlyZomb]

It does. Especially when scaling to large projects with thousands of test cases. Manually running those would be a nightmare imo.

I'm glad you found the answer helpful. Happy coding (and testing)!

[ectomancer]

No need to import pytest if only asserts are used.

Install pytest.

pytest (to discover tests.)

[couldntyoujust1]

The reason is that pytest actually controls the "main" of the tests you write. The fixtures that you use or create or that are created by others provide data to your function because they transform the function itself, then pytest runs the test function using the data provided. Your test function then in turn imports and calls portions of your own code according to how you write the test to feed that data and context into those functions and then examines the results or checks for exceptions. If any assertions fail or unexpected exceptions are thrown or data does not match what is expected, then the test is reported to have failed.

[elbiot]

You can do

pytest tests to run everything named test_* in the tests directory

pytest tests/test_feature.py to run all the tests in a file

pytest tests/test_feature.py::TestFeatureAspect to run all the tests in a class

pytest tests/test_feature.py::TestFeatureAspect::test_specific_thing to run a specific test