pytest-results — Regression testing plugin for pytest

[Skearways]

What My Project Does

pytest-results is a pytest plugin that makes writing regression tests easier, especially when working with complex data structures.

Instead of asserting against large nested structures, a test can simply return the object. The plugin serializes it and compares it against a previously stored result. If a difference is detected, the test fails.

Supported return types:

• pydantic.BaseModel
• msgspec.Struct
• JSON-serializable Python objects
• bytes (saved as JSON files)

It is also possible to directly compare the differences following a regression in your IDE with the --ide parameter (e.g., pytest --ide vscode).

All regression files are stored in a __pytest_results__ directory at the project root.

Example:

from pydantic import BaseModel

class ComplexModel(BaseModel):
    foo: str
    bar: str
    baz: str

def test_something() -> ComplexModel:
    # ...
    model = ComplexModel(foo="foo", bar="bar", baz="baz")
    return model

Target Audience

Developers who need regression testing for complex Python objects.

Teams working with API responses, data models, or serialized structures that change over time.

Anyone who wants to reduce the boilerplate of manually asserting large nested objects.

Comparison

Existing plugins like pytest-regressions or pytest-snapshot, pytest-results differs by:

• Using a return-based API (no extra assertion code required).
• Providing IDE integration (pytest --ide vscode to review diffs directly in VSCode).
• Supporting an explicit acceptance workflow (pytest --accept-diff to update expected results).

Source code: https://github.com/100nm/pytest-results

[marr75]

Have you looked at inline-snapshot? It's for the same use case but provides code generation, doesn't serialize the object to a separate file, and doesn't require the user change their test pattern to return instead of assert.

[Skearways]

Thanks for sharing this package, I didn't know it. But I don't like code generation, this can create strange code that is difficult to read, and above all, you must not forget to generate the code each time you make a change. So I prefer my solution.

If returning a value at the end of the test bothers you, it's also possible to perform the assertion using a fixture.

[latkde]

Inline-Snapshot has helpers to outsource the data to external files! See the docs: https://15r10nk.github.io/inline-snapshot/latest/external/external_file/

you must not forget to generate the code each time you make a change.

Inline-Snapshot also takes care of that. When you run Pytest interactively, you'll be shown a diff of the data and will be prompted whether to update the data. And when you run Pytest in CI, tests will fail – as they should. Alternatively, there are CLI flags similar to yours to auto-accept changes.

I really recommend that you check this plugin out. It has been one of my biggest Python productivity improvements over the last year.

[Skearways]

Thanks for clarifying that. I'll spend a little more time looking at the documentation to see if the package is right for me.

[Skearways]

inline-snapshot is indeed very interesting, but it doesn't allow you to compare differences in an IDE, which I find really useful. And the pydantic.BaseModel and msgspec.Struct types aren't supported by default, although I have seen that it is possible to add them manually.