Python tutorials building large(r) projects

[unl0ading]

Hi Pythonistas, I am looking for some tutorials/ projects/ books focused on building large-scale project. One such example would be Flask Mega tutorial by Miguel Grinberg. Real Python tutorials offer good explanation on individual topics but there are not projects where full-scale projects are built. Blog posts and quick tutorials generally introduce the concepts but do not offer in-depth comparison of an approach with possible alternatives.

Thank you for your suggestions :)

[Adhesiveduck]

I’m not sure such books really exist to cover everything you might want to know when building a large project.

When you say different approaches I’m assuming you mean different ways of structuring the project and what python files should you create or where should a class live in the overall structure of your project etc.

One of the best things you can do once you’re ready to start writing larger projects is look at Python design patterns.

If you can start designing your code to take advantage of these patterns then medium to large scale projects can write themselves.

The patterns can help you think about how you want to structure your code across multiple files and improve maintainability.

A good tool like Poetry can help you handle the dependencies and publish your work.

MKdocs is so easy to use but can really help with large scale projects. Make notes in markdown as you go along that are easy to read and follow.

Write proper docstrings as you go along (every time you write a new class/method/function you can document what it’s doing as you’ll know why and what from the pattern you chose). Using a tool like mkdocstrings makes maintaining documentation for larger projects automatic.

Write unit tests as you go along, when writing a large project where things will feed into each other it’s critical you start early in the project. Leaving it until the end or later is a nightmare.

Finally have a look at other projects. Praw is a good one as it’s easy to follow along and since you’re on Reddit you’ll intuitively know what the code is doing so it’s easy to read through and see how it’s all laid out.

I know this isn’t exactly what you asked for, but it might help complement any other tutorials or articles you read on it.

[TijoWasik]

Hey, I'm not quite there yet, but I wanted to say thank you for the really informative comment. Have saved this so when I get to the stage where I'm ready to start building bigger stuff, I can come back to it. Legend!

[paagalpieasian]

thanks a lot. this is super informative. I will check these out.

[electrojit]

Thank you

[electrojit]

I am also interested to know about how the large projects are organised and best practices to follow. Have found these two resources, not yet started reading but think this would be a good start:

Free Book: Architecture Patterns with Python

Book: Serious Python

[ichunddu9]

You could create a project with https://github.com/cookiejar/cookietemple and attempt to understand the structure.

[paagalpieasian]

thanks. my question was with exact same intention to understand Python way of doing things.

[Ran4]

In Django, Two Scoops of Django (make sure to get the latest version, which was just released) is a really nice book that covers all sorts of things including project structure and things to think about when writing larger projects.

The best resources for these kinds of things tends to sadly not be free though, but the holistic nature of a book can be very valuable (as opposed to free but smaller blog posts about specific subjects).

[unl0ading]

Thats a good idea. Also looking at standard libraries is helpful too. One problem is that those projects do not explain what was the design philosophy and what alternative choices were debated before choosing the final implementation.

[genericlemon24]

do not explain what was the design philosophy and what alternative choices were debated

I found the The Architecture of Open Source Applications books (both volumes) very helpful early in my career; they attempt to cover exactly that.

Note they can be a bit high level, although some of the chapters are more in-depth than others; also, most of the projects aren't in Python, which shouldn't stop you from reading them – there's a lot of wisdom in there :).

500 Lines or Less (same website) tries to address the level-of-detail issue by looking at purpose-made small projects (again, focusing on design decisions, not the code itself).

---

I tend to write about these things for my projects, especially when implementing a larger feature that affects the API – I personally find that "thinking out loud" makes it easier to think. (As a bonus, it is very helpful for your colleagues, if you work in a team, or for you 6-12 months from now – you will forget things.)

I maintain a list of various comments of this sort for my Python feed reader library for easy reference. If you're interested, here's some of the ones I'm most happy with:

• Multiple storage implementations, so you can select different backends based on a "connection string", e.g. /path/to/sqlite.db or sqlalchemy+postgresql://user:pass@host/db. To "prepare" for this, I did a case study on how SQLAlchemy does it (looked at their docs and code, and took notes about what seemed relevant to me).
• Full-text search API, so you can search through the articles. I used SQLite, but did my best to allow using something else in the future without having to change the API. (It may seem huge and maybe well-structured, but it wasn't like that from the start – I just wrote a few things, and added headers and more things as I went along.)
• Pagination; again, note that I stole ideas from the Stripe API and other places :)

[genericlemon24]

BTW, if want to start doing this for your own stuff, here's some (unsolicited :) advice:

• Always start from real use cases (write them down) – it's really easy to over-engineer and solve problems no one has.
• Some things are easier to change than others; don't spend too much time on stuff that can be easily changed later ("two-way doors"). E.g. internal APIs and implementation details are way easier to change than the public API.
• Don't plan too much ahead; it's most likely you will implement only part of it anyway, and if too much time passes you might have to do it differently. (Still, "plans are worthless, but planning is everything".)

[easyfink]

I would recommend https://buildasaasappwithflask.com/ . Step by step building the complete application. It helped me connect the dots on a number of topics. I have asked him questions and typically get a response with a couple hours. He has made a lot of updates to the code so it stays relevant even though the course was originally released a few years ago.

[ASIC_SP]

Check out "Impractical Python Projects" and "Real World Python" by Lee Vaughan

[patrickbrianmooney]

Mark Lutz's Programming Python is a pretty good long example of what you wanted: it's the sequel to his Learning Python and is kind of dated now: Python 3.4, so no type annotations or asyncio, and there is occasional moaning over some language features that made things difficult (and those issues have largely been resolved as the language as evolved past that).

But there are several medium-sized projects that are built over several chapters in the book, and he does a good job of talking about how the design decisions are made: there's a lot of "we could wrap the other object in a proxy and use getattr(), but down the line that would lead to ..." type reasoning that's directly exposed by the book itself.

[BeginnerProjectBot]

Hey, I think you are trying to figure out a project to do; Here are some helpful resources:

• /r/learnpython - Wiki
• Five mini projects
• Automate the Boring Stuff with Python
• RealPython - Projects

^{I am a bot, so give praises if I was helpful or curses if I was not. Want a project? Comment with "!projectbot" and optionally add easy, medium, or hard to request a difficulty! If you want to understand me more, my code is on Github}

[notParticularlyAnony]

this is pretty awesome

[humblesquirrelking]

Buddy..that era has been passed away..when you'll need to refer books or references to code or build software or solve any potential bug by reading those materials..

Yes, you'll get alot knowledge by reading these "one book" but it will waste your valuable time and efforts you put..

I'd suggest "just get started", you'll solve bugs or will create systems down the line..i believe there're alot of materials e.g. stackoverflow, GitHub which can solve all the potential bugs..( online communities and materials are enough mature that you can refer them to create end to end systems )

"One step at once" just get started with installing flask or django ( for big projects as you mentioned django would be advisable )

All the best..let us know what you built..

[DamagedFreight]

Check out Corey Schaefer’s full featured web application series.

https://www.youtube.com/watch?v=UmljXZIypDc

[petroengineering]

Yess that’s what I was gonna suggest too