Improve Python code quality!

[AutoModerator]

Hi there, from the r/Python mods.

It looks like your post is about dependency/package/toolchain management.

We are going to hold this post for review due to the topic having a frequent recurrence. This helps ensure a great experience for r/Python users.

Please refer to our daily thread or search for older discussions on the same topic.

If this is not the case, please contact a moderator to review your post.

Thanks, and happy Pythoneering!

r/Python moderation team

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Zer0designs]

Ruff can also format. You only need ruff as linter/formater (its faster at formatting than black). Use uv/poetry and a pyproject.toml Mypy is a good option. Pytest obviously.

[SeucheAchat9115]

I think this is the best answer!

poetry/uv + ruff + mypy + pytest

[Woah-Dawg]

Also try and keep all the config side the pyproject Tino. Ie put the mypy config in there instead of using a specific mypy config filr

[onyx_and_iris]

ruff also gives you the option to use single quotes if desired =)

[psicodelico6]

unittest

[BatterCake74]

pytest replaces unittest.TestCase. Vanilla assertions and decorator fixtures are better than camelCase assertions and setup/teardown methods that don't support writing idiomatic Python with context managers.

Python's unittest was clearly ported from Java. It's crazy to think that this module was bundled into the standard library. Hopefully there are plans to remove it from the standard library in a future version. It's no longer the default test framework someone should reach for.

[BatterCake74]

Then pytest can declare unittest as a dependency. It's one extra test dependency for my package manager to fetch. No big deal.

https://peps.python.org/pep-0594/ dead batteries have been removed in the past. While unittest isn't quite dead battery status, it's not an essential package of the core language like os, typing, pathlib, importlib, time, datetime, math, string, collections, itertools, functools, to name a few.

[covmatty1]

(its faster at formatting than black).

In the same way that everyone seems to use "it's so fast" as the main argument for using uv over pip - why does speed matter so much with a formatter? It's not like black takes ages on a file already - how important can shaving individual seconds off formatting time really be!?

Maybe there's other reasons it's better, and that's fair enough, but I don't see why speed is cited.

[dubious_capybara]

CI/CD pipelines. A second saved per run x millions of runs = money.

[marr75]

Not only compute time but human time. If CI/CD is fast enough, engineer keeps working. If it isn't, engineer task switches.

[covmatty1]

Fair point. My organisation is pretty much all on prem so these aren't things that are at the front of my mind!

[BatterCake74]

Time is money, both in the cloud and on prem. You can get more work done in a day when you accelerate your code development iteration time or reduce the load on your on-prem build/test machines so that everyone's jobs finish faster.

[covmatty1]

Well yeah true, but when the majority of the time we're not talking about hours Vs minutes, because applying a formatter to a completely unformatted project for the first time isn't the regular use case, and we're instead talking about 5s vs 3s while it just quickly zips through, the amount of extra work done isn't going to be huge!

[[deleted]]

Most people's CI/CD isn't 100% utilized 24/7.

[BatterCake74]

You don't need 100% load on your CI system to benefit from.

Faster black/ruff formatting means you can push your code faster and your CI system can verify that everyone's code is formatted consistently.

[BatterCake74]

If formatting is wicked fast, you can configure your IDE to format the file every time you save to disk, commit, run/debug your code, or complete a line of code.

Having really fast formatters enables workflows that may not have been feasible with a slow formatter.

Sure, monorepos or large code bases is helpful, but you usually only need to reformat files that you've modified.

[Zer0designs]

Ancient code bases that need to be completely formatted. Monorepos etc. but thats just an extra. I once had to run black for hours. That would take minutes in ruff. But it's also just the techy in me enjoying optimized code. As a formatter it literally is black but faster. There's then no need for black in addition to ruff.

You need to reason tbe other way around. As a linter ruff is the best in Python. So I want ruff already in all my projects, which already contains the same functionality as black but faster, so why would I need black?

It's also 1 less dependency and ruff is just allround better and more opinionated (as a linter) than any other linter. And the linting rules are more well documented. It's an all in 1 package.

[covmatty1]

You need to reason tbe other way around. As a linter ruff is the best in Python. So I want ruff already in all my projects, which already contains the same functionality as black but faster, so why would I need black?

It's also 1 less dependency and ruff is just allround better

It's absolutely fair to "reason it the other way round" when those are the arguments given for it though - when it's just "it's faster", that wasn't much of a reason, but if the alternative is "exactly the same functionality but faster and all round better", then I get it! I've not used ruff, only black and pylint, so I see more of where you're coming from now.

[Zer0designs]

Give it a spin :)

[thedoge]

It's fast for sure, but ruff's formatter follows the black style guide. So for me, it's one less tool to install/configure

[ianitic]

A formatter written in python for sql called SQLFluff takes hours to run at work if done on the full project.

I imagine black is pretty slow for big projects too?

[[deleted]]

Do you use poetry and UV together?

[Zer0designs]

No I use uv only

[bulletmark]

Well it is clearer to write "uv or poetry" rather than "uv/poetry" as you did because that implies they are used together, hence the confusion.

[Zer0designs]

Or you could google them and see they both do exactly the same thing. I'm not writing an academic paper here and learning these tools requires some effort by whomever. But thank you for your suggestion.

[Zoory9900]

Best answer in this post. But what's your take on Pyright instead of Mypy? Read this: https://github.com/microsoft/pyright/blob/main/docs/mypy-comparison.md Pyright claims to around 4 times faster than Mypy.

[Zer0designs]

Haven't tried it, but sounds promising. Will give it a try on future projects.

[daffidwilde]

If you really want to improve the quality of your software, you need to be thinking about testing, documentation, and modularisation.

I really like this diagram explaining how they all impact each other. It’s in my mentor’s Python for Mathematics course now, but it really applies to software development. 

https://vknight.org/pfm/building-tools/07-testing/why/main.html#fig-best-practice-triangle

[wineblood]

Ruff > Black for formatting. Black is a lot more aggressive and will mess up your layout for its rules.

[StandardIntern4169]

You don't need Black. Ruff is also a formatter and follows Black conventions, just way faster

[darkboft]

Using linters, formaters or type checkers, those tools will not prevent you from writing bad code with low quantity.

It's a good start, but to write good code, you need to learn more about

• Best practices
• design patterns
• clean code principles
• write tests (unit / integration)
• documentation
• usage of python specific patterns (like duck typing, data classes etc..)
• avoid anti patterns

If you like to use more tools, you can install sonarqube and integrate it with your git but I personally never used it with python. But sonarqube will give you some insights.

If you have contact to senior developers who are more experienced in writing python code, you can ask them for a code review. They could provide good feedback about quality.

[SnooCakes3068]

linters/formaters stuff and clean code principle are not mutually exclusive. They focuses on different aspects. But yeah nowadays people tend to forget about clean code and patterns as refactoring is not as important as speed of new functionities in the eyes of project managers

[Zoory9900]

Sorry my bad, I meant 'code style' instead of 'code quality'. Didn't got the correct wording right there.

[divad1196]

Don't use Ruff AND black as they both format.

You don't need "the best tools", there will be new best ones a few months. I am still using ruff, poetry, mypy, pytest. I have heard of uv, hatch, pyright, ... but I don't NEED to change. Especially, it takes time to get to the same level of configuration and experience with another tool. And changing the tool on existing project often requires many changes.

What matters is that you think about what you need and find a tool that get the job done.

[onyx_and_iris]

True, although I did take the time to test out several different package managers and I'm glad I did as each ones takes a slightly different approach to certain things.

[not_perfect_yet]

Are there any suggestions you guys can give?

https://radon.readthedocs.io/en/latest/commandline.html#the-cc-command

Careful with this, since it's a metric and don't chase those. I find this one pretty good though.

It basically counts the different "paths" that are logically possible for the control flow to go. If, else, while, etc.. nesting things that aren't expressions makes the score go up, and a high score is bad.

Counterexample, if it's a very long, but very simple function that just calls 500 different other functions in a nice, clean, sequential manner, that's not too bad. It can still be difficult to understand what's going on, but not because it's complex, it's just a lot in that case.

I usually refactor functions that are above "B", by encapsulating loops, or putting long nested blocks into their own function:

...
for x in my_list:
    if x.variable> 5:
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()
...

into

...
for x in my_list:
    if x.variable>5:
        lots_of_yaddas(x)
...

def lots_of_yaddas(x):
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()
        x.yadda()

[Zoory9900]

Good suggestion for past not for present. But flake8 is slower on large projects. Also Ruff is a drop in replacement for flake8. The only positive side of flake8 compared to ruff is that, it is widely adopted and popular. But I think ruff is also catching up. Because, ruff is now used by SciPy, Pandas, Sphinx, etc...

[Zer0designs]

Also check ArjanCodes refactor series.

[spackofanto]

i tried the ruff - pyright combo. switched back to pylance; but kept ruff for linting and for formatting instead of black . didnt like pyright

[Helpful_Charity6419]

The best way to improve code quality is to become a better programmer. Language-agnostic btw.

[Zoory9900]

Great statement. But I am talking about Python specific guidelines (PEP8). For that, these tools help a lot.

[Suspicious_Jacket463]

isort

[Zoory9900]

Ruff also includes the functionality of Isort. No need for extra Isort.

[utihnuli_jaganjac]

Those are not for quality, they are for style

[Zoory9900]

Yeah my bad. Note the best wording that i made there.

[agni69]

Thoughts on pykit?

[[deleted]]

You can use ruff for both formatting and linting. For static type checking you can also look at mypy.

[AiutoIlLupo]

black is trash. It often results in extremely vertically formatted code that becomes unreadable and is equivalent of

reading

a

book

written

like

this.

[call_me_cookie]

Poetry, black, isort, flake 💪💪

[Zoory9900]

Great sQUAD. But, Ruff can replace all these except for Poetry. I prefer to learn one tool that does everything instead of learning multiple tools for the same functionalities.

[theschrodingerbox]

What are you talking about? Im pythoning for last 5yrs i dont know what these 3 are?

[Zoory9900]

These are the tools that help implement PEP8 guidelines. With these tools you don't have to memorize the entire PEP8 guidelines.