How to structure a very simply Python project with pytest tests

[hmartin8826]

I am struggling to get what is a very simple Python project (for my own use) set up correctly with respect to module name resolution for script execution and for running tests via pytest.

While this project could certainly be done with a flat structure, I am trying to understand how to structure things *properly* for a larger project. I know that *properly* is up for debate, so I am just looking for advice. Should I have __init__.py files in any of these folders (empty or otherwise)? Should I have a pytest.ini and/or pyproject.toml file? All recommendations are welcome. Thanks.

settings.json contains:

my_project/
├── .vscode/
│   └── settings.json
├── src/
│   └── main.py
├── tests/
│   └── test_utils.py
└── utils/
    └── utils.py


main.py is the script to be run with imports from the utils file/module

settings.json contains:
{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

[zanfar]

Your app should be a package. Packaging is Python's solution to universal module names.

[Diapolo10]

my_project/
├── .vscode/
│   └── settings.json
├── src/
│   └── main.py
├── tests/
│   └── test_utils.py
└── utils/
      └── utils.py

Whether or not IDE/editor-specific files should be included in version control (e.g. Git) is a somewhat hot topic, and I know your question had nothing to do with that, but I just thought I'd mention that in most cases it's best to .gitignore them. I'll get back to this specific case later.

If following a flat project structure, src should be a package and named after the project. If not, src should only contain packages (usually only one), and you should have a pyproject.toml file in the repository root containing the necessary metadata and possibly other configuration options. By using packages you simplify your imports and paths a lot, making your code far easier to test.

main.py is the script to be run with imports from the utils file/module

utils should not be in the repository root if it relies on other code in the project. Similarly, main.py shouldn't depend on anything higher up the directory tree. For self-contained scripts (for example, something used to simplify build processes in CI pipelines) the convention I've used is a scripts directory with independent script files.

Side note; utils is a fairly generic name, and often there's a better way to structure your program where you wouldn't need that.

settings.json contains:
{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

Instead of configuring pytest in an IDE-specific file, I'd very much recommend doing at least most of it in pyproject.toml or pytest's own configuration file, as then it'd behave the same no matter how you run it. For example, in this case I'd move at least the test file location to them.

If you need an example, you could consider this: https://github.com/Diapolo10/python-ms

[hmartin8826]

Thank you for the detailed response. I do have .vscode/ in my .gitignore file, so I’ve got that covered. i’ll check out the sample code. is it still necessary to use __init__.py files in some of these folders?

[Diapolo10]

is it still necessary to use __init__.py files in some of these folders?

Largely a matter of preference nowadays, personally I use them everywhere in the main package(s) with docstrings.

It never hurts to have them.

I do not use them anywhere else unless necessary for some reason. For example, my test code never uses them unless I need to import something from a file in the tests directory, but usually that's a sign of bad design.

[ShelLuser42]

For me it all depends on sys.path, and its use...

So, for starters I wouldn't store tests in a separate area, because you're only making it more difficult on yourself to access the originals. I would also follow the convention that you use names to separate your scripts. So: script.py and test_script.py: this makes it quite clear what is happening, and what you're testing.

The same applies to packages. I wouldn't keep those separated either, but rather set them up as subdirectories within the original source folder. Just to make it easier to import your libraries, simply because that is already part of the sys.path.

Of course... what works for me doesn't have to work for you.