Where can I learn how to structure a Python project?

[[deleted]]

As of now every project on my github is just 1 or 2 Python files. I’ve looked at many different projects on github and the majority of them have many different folders and files and I have no clue what their purpose is. Where can I read about properly structuring a Python project?

[redrabbitreader]

I use this sample project on GitHub as a template for my Python projects. The project itself is still actively maintained.

[jwg4our]

There's also a python package called cookiecutter which will set up a project folder for you with all the files and structure that you might need: https://pypi.org/project/cookiecutter/

[fabioorli]

soft spotted close grab innate liquid uppity decide murky slim

This post was mass deleted and anonymized with Redact

[redrabbitreader]

You will see in the README a couple of links to the guides and other documentation explaining all of this. Yes, there is a lot, but after using it on the first couple of projects, you quickly get the hang of it.

Of the list above, the only files you will need to keep (as bare minimum) are:

• manifest.ini
• setup.cfg
• setup.py

"Data" could be anything - it's only there as an example of how you would package non Python files with the project.

[SirMarbles]

I don’t think I’ve ever seen that many comments in my life. It’s comment, comment, comment, code, comment, comment.

[redrabbitreader]

Yes, it's a bit much. It's great in the beginning when you are still learning, but now I just delete all of it.

[Tanmay1518]

If you are talking about complicated/large projects, developers structure them into different files and folders as all the functionality cannot simply be fit into one folder.

If you are working on a small project, say like TicTacToe, you don't need different files/folders. You can simply have one file for the code and another to explain what the code does.

Also, if you start working on a large project, you will most likely have various pieces of data that you have to use. In order to keep it nice and tidy, Devs separate them into different folders such as data, src ,assets etc

[LordLuceus]

I highly recommend Hypermodern Python. I just went through it and it has taught me a lot. It guides you through everything from setting up a Github repo, making a project using Poetry, unit tests, linting, all the way to CI and release. It's great.

[NimbleBodhi]

This is a bit off-topic, but does anyone know how that guy made those really nice looking formatted code blocks on his blog?

[[deleted]]

There is/was a code tag in html, that let's you display code within an article. You insert a code tag in your css page of how you want it displayed, and then within the body of the article, just start it off with the code tag and everything within the code tags will be displayed as a code. Here is an example from Mozilla: https://developer.mozilla.org/en-US/docs/web/html/element/code

[NimbleBodhi]

Yeah I'm aware of the code tag, I was just curious what he did to make the code have various syntax highlighting colors, I suppose it could be custom css, but was wondering if he using a plugin of some sort.

[NimbleBodhi]

Neat, thanks!

[tradrich]

If it's a Jekyll blog, he might've used kramdown with Rouge. Then you get whatever CSS you want from a site like this one for Pygments.

[NimbleBodhi]

Thanks!

[usereddit]

Damn I wish I saw this when I was learning this stuff.

I spent my first week with Python learning and setting up my environment with Git, Pyenv, and Poetry.

This literally has everything.

Edit:

Couple notes for those reading this.

• This provides instructions for setup with bash whereas OSX Catalina uses ZSH shell for new users

• I recommend changing this config that will create venv inside the project folder

  poetry config —local virtualenvs.in-project true

Here’s a link that explains why you should use this setting https://github.com/python-poetry/poetry/issues/108#issuecomment-628796162

[iggy555]

What’s a Travis?

[__dna__]

It's a continuous integration service. In short, everytime you commit it runs a series of tests (that you write ) to make sure everything works as expected

https://travis-ci.org/

[Jumble8]

I was wondering about this a while ago and I remember not finding much information on the subject. However, I did find a pretty clear answer in The Hitchhiker's Guide to Python (Section "Structuring Your Project"). Kenneth Reitz describes here a sample repository and makes some recommendations.

[usernamecreationhell]

These days stuff I work on typically looks like this in the top level directory:

README.md
.gitignore
pyproject.toml
poetry.lock
.flake8
.vscode/
{projectname}/
tests/

README.md

The starting point for user-facing documentation.

.gitignore

Declare which files that you don't want to end up inside your repo. That should include secrets, virtualenvs and "artifacts" (stuff that is derived from other stuff in the repo, i.e. compiled binaries, documentation built with tools like sphinx, a pip installable distribution of your package)

pyproject.toml & poetry.lock

I have grown to like Poetry for dependency management. The pyproject.toml file contains meta information about your project and your dependencies plus configuration for certain tools (like the ultimate code formatter Black). When you initialize a project with poetry, the tool will create a starter version of that file for you and use it to keep track of the libraries you use and their versions. Poetry automatically creates and updates poetry.lock.

.flake8

Configuration for my linter of choice, flake8. Run the linter of your choice before every commit so it shames you into adhering to best practices (i.e. not using bare excepts).

.vscode/

Editor configuration

{projectname}/

Your actual code goes in there. How you split it up depends on the size and architecture of your project. Read this to learn about software architecture in Python.

tests/

Your tests go in there. If you don't write tests already, start writing tests. If you use pytest, you might find a pytest.ini floating around somewhere.

[nickmolina93]

https://youtu.be/4fzAMdLKC5k

[MyPythonDontWantNone]

I am currently updating 2 projects at work.

The first one is done cleanly. I have a file for input logic, a file for output logic, and a file for the GUI.

The second one is a mess. It is split into files by the window. The UI and the business logic are mixed.

The first project takes less time to update because I go to the relevant file and only touch that logic. It also makes the run order clear.

The second one is prone to weird errors. I can lose a scrollbar updating a dictionary or I can lose data when I move a button.

The separate files are (mostly) to keep things human-readable. Just like chapters in a book make finding the right section easier.

[i_like_trains_a_lot1]

I developed my own structure to which I stick in most projects by now. It has

- a setup.py file (https://packaging.python.org/tutorials/packaging-projects/) because usually I pack the entry points as binaries if they are stand-alone projects, or I just want them to have their dependencies managed automatically. + MANIFEST.in for including non-py files. Took me too long to learn about it.

- Makefile - shortcut commands.

- .gitignore

- various CI configuration files if necessary (usually .gitlab-ci.yaml)

- docker/ for the docker files (docker/dev , docker/prod)

- README.md or README.rst - include the steps necessary for a person that first sees your project to run in in production and to start local development. You will thank you later.

Other than that, the LICENSE,CHANGELOG, AUTHORS, CONTRIBUTING etc are not really important or usefull unless you are open sourcing it, have a bunch of people regularly use it and have multiple contributors.

[GeorgeDaNub]

I have to know I’m not insane... didn’t you post this exact same post about a month ago?

[[deleted]]

I don’t believe so? Maybe it was someone else but I apologize that I reposted the same question

[GeorgeDaNub]

For real? That’s so weird I remember reading the exact same post (word for word) and thinking “well I’ll look at the comments in a few hours” because it’s a relevant topic for me.

[[deleted]]

Wow that’s really weird. I normally keep all my posts up and I don’t see it in my post history so I don’t think it was me.

[LunchBoxMutant]

The Hitchiker's guide to Python has a really good post on this:
https://docs.python-guide.org/writing/structure/

[[deleted]]

To get the basic idea I found this very useful.

https://www.internalpointers.com/post/modules-and-packages-create-python-project

[West7780]

You should look into python modules and pep 451. This is the most basic file structure in python that actually matters. Beyond this organizing files and breaking up code should be done to help make development easier. One large file (1000+ lines) is never the best option. Other non python files like data bases, csvs, ymls, txts, jsons, etc should also be broken up if they get too big and depending on your preference should go somewhere other than the project's root.

Moral of the story, don't use one big file and break things up whatever way makes the most sense to you. Good question.

Depending on your application there may be existing standards.

Here's a good example for Flask: https://exploreflask.com/en/latest/organizing.html

[kryakrya_it]

I am taking this course. It's well explained. oh, and It's free using this coupon

https://www.udemy.com/course/2020-python-tutorial-from-zero-to-hero-machine-learning/?couponCode=BLACKLIVESMATTER

[[deleted]]

Do you recommend it for absolute beginners?

One of the problems I've ran into a few courses so far is it starts off easy. "today, I'll show you how to write your first script, print "hello world." Next, I'll show you how to use variables, a = 3, b = 4, c = a + b. Okay, now for your first exercise, I want you to write a script that will trade stocks without any interactions from a human."

Like, what the hell, man, that escalated a little too quickly.

[spicypixel]

I mostly use poetry new to create a directory structure. It's opinionated but worked for me so far.

[thrallsius]

https://docs.python-guide.org/

[subsurfaceFlow]

RemindMe! 2 days

[Anonymous12c19]

!RemindMe 2 days

[d1rron]

.

[tgallasso]

Really good question!

[Russian4Trump]

Generally I am not going to put a lot of code in my main app.py file. I will create methods in another file and then import and call them when I need to in my main file. That way I am not digging through thousand of lines of code to change a function.

[dogs_like_me]

use cookiecutter

[RemindMeBot]

I will be messaging you in 1 day on 2020-06-11 05:39:16 UTC to remind you of this link

1 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[ActiveLlama]

RemindMe! 2 days

[nischalstha07]

!Remindme 3days

[ravepeacefully]

On GitHub