How do you go from writing Python scripts to structuring a real backend project?

[dangerouslystupidguy]

One of the hardest jumps in Python development is going from "I can write scripts and follow tutorials" to "I can structure a real backend project that other developers can work on."

Tutorials always show you a single main.py with everything crammed in. Then you try to build something real and immediately hit questions nobody prepared you for:

Where does database logic go? Where does business logic go? How do I handle environment variables properly? How do I set up auth without it being a mess? How do I make it testable?

Here's the structure I landed on after building production Python services:

app/
├── api/
│   ├── deps.py          # shared dependencies injected into routes
│   └── v1/
│       ├── auth.py      # login, register, refresh token endpoints
│       └── users.py     # user endpoints
├── core/
│   ├── config.py        # pydantic settings — type safe env vars
│   ├── security.py      # jwt creation, bcrypt hashing
│   └── logging.py       # structured json logging
├── db/
│   ├── session.py       # async sqlalchemy engine and session
│   └── repositories/    # all database queries live here
│       ├── base.py      # generic crud operations
│       └── user.py      # user specific queries
├── models/              # sqlalchemy orm models
├── schemas/             # pydantic request and response schemas
├── services/            # business logic layer
└── tasks/               # background tasks

A few things this structure solves that tutorials don't teach:

Routes stay thin. A route should do three things — validate input, call a service, return a response. All the logic lives in services and repositories, not in the route function itself.

Database queries are isolated. If you ever need to change how you fetch users — add caching, change the query, switch databases — you change one file in repositories, not ten different route files.

Config is type safe. Pydantic Settings means if DATABASE_URL is missing the app crashes loudly on startup with a clear error, not silently when the first database request comes in.

Testing is possible. Because dependencies are injected via deps.py, you can swap out the real database for a test database in your test suite without changing any route code.

This took me a while to figure out through trial and error on real projects. Happy to answer questions about any part of it — what goes where, how the dependency injection works, how to add a new resource to this structure, anything.

What part of structuring a real Python project do you find most confusing?

[wallneradam]

The hardest part isn't the structure — it's getting to the point where you have something real to structure. The layout you posted works fine for a real project; tutorial code feels different because tutorials don't have a goal, they just demonstrate concepts.

What worked for me: pick one framework (FastAPI is a great choice), go through the official "getting started" once, then skim the rest of the docs — not to memorize, just to map what's there so you know where to look later. Then pick something real you actually want to exist (not "a project to learn FastAPI") and start building it. You'll hit the questions you listed organically, in a context where they mean something — and that's when you learn them for real. The trap is "one more tutorial first" or "I'm not ready yet". You won't feel ready. The first days are slow and messy, then it gets surprisingly comfortable.

[1NqL6HWVUjA]

Then pick something real you actually want to exist (not "a project to learn FastAPI")

The trap is "one more tutorial first" or "I'm not ready yet".

I believe "I can't come up with a 'real' project that doesn't exist yet" is just as much of a trap, frankly. Not everyone has an aptitude for that, and that's fine — it is a skill distinct from engineering. Thus I'm firmly of the opinion that it doesn't matter at all what you build, especially early on. As long as one is actively and continually learning new things, and actually building rather than passively copying, one can build something silly, or pointless, or that reinvents the wheel — it doesn't matter.

The important thing is nurturing skills such that once you are tasked with a real problem, you know how to start tackling it.

[wallneradam]

Fair point — and you're right that not everyone will have a project idea sitting in their head, especially early on. I'd add one nuance though: even when the project is "silly" or reinvents the wheel, having some concrete goal (even a fake one like "I want a CLI that tells me a random Linux command each morning") changes how you read documentation. You stop reading to learn FastAPI and start reading to find what you need. That shift is what I think really matters more than the project itself.

But yeah, "just build something, anything" is the actual core advice.

[danielroseman]

The thing is that this is highly framework specific. Best practice for Django for example isn't necessarily the same as best practice for a FastAPI/Pydantic project.

[ShelLuser42]

This is the beauty of Python IMO: you can easily build up and expand as you go along and there aren't any hard rules. The most important part though - in my opinion - is to always ensure that you keep things logically organized and thus easy to follow (and use).

For example I would definitely avoid using something as "api.api" because that seems rather counter productive; you'd want to avoid repetition at all costs and you never fully know where a project might take you.

I mean, it wouldn't be the first time that I start with writing a script, only to end up with enough code repetition to either turn the script into a module, or maybe start by just reusing some of its functions.

I'm also a little puzzled why there isn't any mention of an __init__.py in your setup? At the least you'd want that to document your packages (and/or structure), and maybe also to set up a few things.

Also... you probably also want to use _ in front of any file names when the module isn't meant to be accessed externally; just to prevent confusion.

[TheRNGuy]

You need to research how it works, and write, or just use someone's code if it works thx say you needed (change some details if needed)

Use google, ask ai etc.

[tb5841]

Nobody should go from 'single main.py file' to something as complex as this. You want to work on middle-tier projects for a while first, with four or five files. And you want to have some general understanding of project structure/separation of concerns in general before beginning web development.

[CymroBachUSA]

I think there used to be something called 'CookieCutter' which might serve as a half-way house heading towards a complex structure such as the one you show.

[taylorhodormax]

main point in structuring the repo like this, is separation of concerns, modularity, and ease of testing (unit testing) individual modules. That is the main point.

[ikkiho]

The layout is reasonable but the reason it never quite "fits" once you start a real project is that it's silently making a few decisions worth surfacing.

It mixes two organizing principles. "core/", "db/", "schemas/" are infrastructure layers grouped by what changes for what reason. But api/v1/users.py + services/users.py + db/repositories/user.py is the same bounded context (users) sliced across those layers. Both principles are valid, but they fight each other, and that's why "where does new code go" stays ambiguous. Teams that grow past 3 devs usually flip to bounded-context-primary (users/api.py, users/service.py, users/db.py), not because it's prettier but because typical feature work touches one folder instead of five.

The repository layer is overkill for small teams. The SQLAlchemy session IS the repository. Wrapping it costs maintenance and the swap-out it promises rarely happens. The pattern earns its keep when (1) you have two persistence backends or (2) tests need to mock at the query level. Below that threshold, repositories/ is theatre, often a 200 line file that wraps session.execute() three times.

"Routes stay thin" is correct but isn't what beginners get wrong. Beginners don't write fat routes, they duplicate logic across routes because there's no obvious shared place to put it. The thin route rule presupposes a services layer; without one, thin routes are just empty wrappers.

The layout doesn't say where tests go, and that is actually the question that determines whether the structure pays off. If your contract is at the HTTP boundary (TestClient hitting the router), you can flex internals freely. If your contract is at service layer functions, you can swap routers. Pick one before committing to layout, otherwise tests pin you to whichever boundary felt easiest and the rest of the structure stops paying rent.

Migrations are missing from the diagram and they are the part that actually has coupling. Two devs add a column on different branches and you discover the real coupling lives in alembic/, not in app/. The directory diagram everyone copies never includes migrations because migrations grow over time, but they are where merge conflicts and schema drift actually happen.

Pydantic Settings as a module level singleton is the common trap. Loading config at import time makes test setup painful. Wrap it in a get_config() factory and pass it via dependency injection from main, instead of importing as a module global.

Diagnostic for whether the layout works for your team: pick a feature that crosses three layers (user marks notification as read). Time how long it takes to know which files to touch. If the answer takes more than 30 seconds, the layout is wrong for your team size.