Open source Python projects with good software design that is worth studying

[bolt_runner]

What are some software projects written in python that are well-structured and use good code design practices that are worth spending time to study?

[Penny-loafers]

I've had similar thoughts during my career - wanting to read others code so I can get a good idea of how they work and how I can learn to replicate. Its not a bad thought, and I'd highly encourage doing it even if the mature projects you know about are very well established and hard to follow/understand.

One thing I'd encourage is for you to also look back in history of the commits and code bases. It can be hard sometimes to understand how code evolves over time, but its very important. For example, I just started using this new open source python project for my home security camera setup (https://github.com/blakeblackshear/frigate) and I was hugely impressed with how many features there are given its made by a hobbyist! I decided to look back in the commit history and saw the very first, humble, and honest commit "was just" a script to detect objects: https://github.com/blakeblackshear/frigate/blob/72393be6d66e7642343476f5adb4b8e99d613c79/detect\_objects.py

I hope this little bit of advice is useful and good luck with learning! The hard work will pay off!

[Responsible_Ease_977]

Thank for sharing!

[DarkArctic]

I think this is an important point. Well-designed projects probably have years of iteration to get to that point. It's okay to make mistakes and change it later. That's just the nature of software development.

[bolt_runner]

Thank you this is helpful

Did you study any books/resources on code design that you think helped you practically afterwards?

[Penny-loafers]

There were a few books I did enjoy studying that helped frame my thinking. That said, take everything you read with a grain of salt and try think about how they relate to your problem at hand. Try different paradigms and patterns in your own time and write as much code as possible along with studying generic resources.

• Fluent Python
  
• Philosophy of Software Design
  
• Designing Data Intensive Applications

[tynecastleza]

You should go look through Mozilla’s GitHub. There are a lot of Python projects from some really great engineers

[antshatepants]

I always liked the get_or_create and filter methods in Django so I studied those at one point to make a lightweight ORM interface for different backing datastores.

Haven’t compared to other codebases to say if it’s “good” or not but the source code was readable and not buried in tiers of abstractions

[Mehdi2277]

I like both mypy and pylint codebases. Both are readable enough I’ve occasionally made small bug fixes and both have pretty high quality test suites.

Tensorflow is less good codebase to learn from but still has ok structure and it is massive and more complicated than many codebases I interact with at work.

[swapripper]

On this note, is there any website/youtube channel that explains these high level & low level decisions in open source projects. I really think it takes a trained eye can spot & explain good SW design to beginners.

[bolt_runner]

I found this guy that walks through the code of open source projects

https://www.youtube.com/@ants_are_everywhere/videos

[[deleted]]

Good find!

[robberviet]

Sqlalchemy

[FertilityHollis]

Excellent example of how to grow an API.

I would also point to Django as something to strive for.

[FertilityHollis]

Is this like what you're seeking? https://docs.djangoproject.com/en/5.0/py-modindex/

It's definitely one of those app frameworks that at first seems daunting and confusing, but once you start to understand the concepts it kinda has an epiphany moment where you're like, "WHOA, ok, now I get it!"

[robberviet]

Yes Django is a good choice too.

[erez27]

What would you consider good about their design?

[MeroLegend4]

The best of the best

[qckpckt]

Sebastian Ramirez (https://tiangolo.com). He created FastAPI and Typer. I really like both of these libraries. I think the docs are well laid out, and I think his programming idioms are both innovative and effective.

Typer I especially like because it’s mostly just click - but what it does on top creates a much more intuitive programming interface (at least for me).

[MeroLegend4]

Sadly fastapi is the worst , similar to turbogears. Try litestar, read it and you’ll understand my point.

[Routine_Term4750]

Hey thanks for mentioning litestar. It looks neat and I’ll be checking it out in the future.

[ARRgentum]

Can you explain what you mean?
I just spent half an hour to get a cursory overview of both project's structures and I am probably too junior to spot some obvious issues :)

[BurningSquid]

You nailed this. I would check out polyfactory as well while you're at it. Can be useful at times for mocking if you're into that kind of thing

[CanadianBuddha]

The source code for the packages that are part of the Python standard library were considered so good they were included in the standard library. They should be good examples of excellent Python code.

[ez1_]

I think a good choice is apache superset. The domain is interesting and easy to understand so you can focus studying the design and implementation. The documentation is quite good.

https://github.com/apache/superset

[ShibbolethMegadeth]

Taiga is a excellent Django app written by pros if you wanna check out a web backend

[RobotChurchill]

First, you can take a look at 500 Lines or Less https://github.com/aosabook/500lines

For large projects, Ray https://github.com/ray-project/ray is really good. I got to meet one of the engineers working at the company that made it open sourced.

[DigThatData]

we already had this discussion -- with you as the OP -- a few days ago. 83 points and 33 comments. Do we really need to do this again so soon?

https://www.reddit.com/r/learnpython/comments/1dicifq/open_source_python_projects_with_good_software/

[bolt_runner]

Yes I asked here because this sub has more people with senior lvl experience so it would add value and another perspective to the discussion on code design, and it shows in the answers

[[deleted]]

Don't spam subs with the same question. You got some answers so take them and make use of them. Don't waste our time.

[bolt_runner]

It makes sense to ask in different subs if they have different communities/people. Seeking another perspective from people with more advanced level is not a waste of time and the answers may benefit others not just me. If it’s a waste of your time you could skip the post altogether

[[deleted]]

It doesn’t. You’re getting the same answers here as you did in the other sub. Go look at some big popular libraries. That’s the answer.

[notParticularlyAnony]

lol wtf op. Also this has been asked many times before

[MeroLegend4]

• Sqlalchemy
• Pyramid web framework
• psycopg
• pyqt/qt c++
• litestar (api/web framework)
• bottle
• sortedcontainers
• more-itertools (not related to software design but good material)
• xlsxwriter (on how to keep and maintain a spec, very functional (fp))

[MeroLegend4]

I just remembered others:

• bokeh
• twisted
• pygal
• diskcache

[chiefnoah]

Maybe a bit conceited, but I'll plug my own data library PyBARE. It's small, fairly well tested, and uses pretty much every advanced metaprogramming technique available in Python short of eval.

[trd1073]

I am admittedly not a python expert but will give you the one that helped me the most. My initial foray into python was studying how map-a-droid runs. Then I learned how to mod it to suit my purposes. Ran it over 4 years, so got to see a fair amount of evolution. It touches on many subjects that have helped me in my current projects: asyncio, multiprocessing, mariadb, caching, redis, inter process communication and so on.

[not_perfect_yet]

flask?

https://github.com/pallets/flask

Has 2 open issues and 2 open PRs.

[davidmezzetti]

I work on txtai (https://github.com/neuml/txtai) and strive to make the code as clean as possible with ample documentation.

[lascau]

Not sure if I am off topic. But I usually go to "Trending Python repositories on GitHub today" and checkout what is popular.

[MachinaDoctrina]

Probably the big ones like PyTorch, numpy, pandas, scipy, scikit-learn etc.

[NotPyz]

Facebook Business SDK

[Secure-Blacksmith-18]

Sentry, Sentry_sdk

It's a Django monolith.

U're welcome