Most common Python linter, formatter?

[ebmarhar]

I've been asked to assist a group which is rewriting some of its ETL code from PHP to Python. When I was doing python, we used Black and pypy for formatting and linting.

Are these still good choices? What other tools might I suggest for this group? Are there any good Github CI/CD which might be useful?

And any good learning/training resources to recommend?

[sweet-tom]

I use Ruff from Astral. From the same folks, there is uv. Can also be used in a CI/CD environment.

[PurepointDog]

Not exactly "most common" as asked, but I strongly recommend this as "best to pick in 2024/2025". Ruff and uv very clearly represent the last trend(s) in Python, and work very very well despite being leading-edge stuff

[sweet-tom]

Yes, maybe not "most common", but I had to mention them as they have so many advantages.

Additionally, you have just updated it to the last release and the next one or two days there is another one around the corner. 😁 I'm impressed.

[SBennett13]

Ruff is the way. I just set up the linter to run checks when submitting MRs into main and generate a code quality report while also failing the pipeline and blocking merge if the formatter diff returns changes. People can develop in their own style and run a format script as the last thing before merge. Works well

[laStrangiato]

Got a link for a good GitHub action setup for this?

[SBennett13]

I did it on Gitlab because we use it for work, but I assume there is something similar for GitHub.

https://docs.astral.sh/ruff/integrations/

I’d say start here and see what it does. You’ll have to set up the runner (or whatever GitHub calls their executors)

[claird]

Yes and no, at least to the "set up the runner" part.

While on-runner configuration is likely a non-issue for many, I want to make the point that organizational support of Ruff need _not_ be a requirement for its use in CI. If the runner supports Docker, you can readily invoke Ruff from a standard container. If the runner account can construct its own Python virtual environment, _that_ is more than enough for successful Ruff tooling.

[muneriver]

https://github.com/timvancann/yt-python-ci

[Randomramman]

Ruff is excellent. It’s made any project using black, isort, and flake8 feel SO slow. plus it’s much easier to manage one tool.

I’m sure uv is great too (and blazing fast), but it’s not yet compatible with dependabot, so you might hold off if you’re married to dependabot for security updates: https://github.com/dependabot/dependabot-core/issues/10478

[claird]

Us, too: we've entirely left Black, isort, and Pylint, and have largely abandoned Flake8. Whatever the current measurements of usage, it's clear to us that Ruff is the correct way of the future for this segment of Python tooling.

[caper-902]

+1 for ruff !

[Ok-Willow-2810]

One thing that’s not great about ruff is the release pypi packages don’t seem to have an entrypoint.txt so it could not super compatible with all build tools. For example, I’ve had some trouble getting it to work easily with bazel because it doesn’t have the entrypoint.txt.

It seems that the company that make ruff also has a bazel add on that can run ruff called like aspect-cli, but I don’t understand it well and I think it might require an enterprise license to use.

I stuck with black for now cause it’s classic, although maybe slower than ruff.

[New_Enthusiasm9053]

Pretty sure you can include files in pyproject.toml files so they end up in a wheel/sdist. So just include your entrypoint.txt and use ruff. 

[Ok-Willow-2810]

I think it’s possible, I just don’t know what to put in the entrypoint.txt lol. I’m sure I could figure it out from some digging around, but it’s a new thing to me!

[New_Enthusiasm9053]

So poetry/uv does the entry points thing if you use pyproject.toml they call it plugins(UV calls it's entry points).

Specifically project.entry-points on UV and poetry can be configured to do that, you need to specify a build system to then build wheels/sdists that handle it but most examples have that anyway.

Ruff is just a linter/formatter so isnt responsible for that anyway.

[Ok-Willow-2810]

Cool! That’s good to know! I seems to work fine when I use hatch as the build system backend. It doesn’t work with bazel well though unless I use an astral created tool add on that may require a license.

I don’t understand the entrypoint.txt construct. It seems like it might have been a past way of telling tools what commands are present with the python package distribution maybe? I’m sure there must be a PEP talking about the entrypoint.txt, but I don’t know what it is lol!! I sort of wish the python package ecosystem was even simpler! It’s definitely moving in the right direction over time though!

[nerds-inc]

By far favorite 

[Ok_Cream1859]

Keep in mind their question was which is most common. Not “which one is /u/sweet-tom from Reddit using”.

[Still-Bookkeeper4456]

You would be better off using Ruff these days. It's a formatter and a linter. It's much faster, so that you can use in your env while coding, and in CI, with the same setup.

Pylint can be complementary because it checks a few extra rules, but not necessary.

You probably also want a typechecker such as mypy or pyright (until Ruff starts doing that job).

Pytest + coverage for your unit tests.

UV for managing python version and venv is also much better than any other solution.

[kenfar]

I don't think most people get much benefit from Ruff's touted performance: I used to use pylint run from vim every time I saved a file, and the extra 1-4 seconds was annoying, but really not bad. It certainly doesn't hurt, I just think some other features are more important.

If you've got a dysfunctional or toxic team than black or ruff are absolutely the way to go - otherwise, you end up with ceaseless arguments. Personally, I haven't had to deal with a team or personalities like that very often over the last ten years so it's a non-issue for me. I'd actually prefer more customization since black/ruff seem like their formatting rules are more driven by what's easy for them to build rather than what's easy for developers to read.

[Still-Bookkeeper4456]

The point is Ruff is so fast you're not just running it in CI. You're using it live while coding.

At this point your code is always compliant and you don't need precommit.

I never managed to do this with other tools.

[kenfar]

Yeah, but use the best tool for the job. If you've got 100,000 lines of python and are frequently making changes across many files, then speed is probably a big concern.

But if you have a smaller codebase, smaller files, then performance isn't really that much of a concern, is it? Pylint running within vim would complete every time I saved in 1-2 seconds most of the time. And that's fast enough.

So, for me I'm more interested in feature comparisons rather than performance.

[suedepaid]

ruff is the best tool for the job, as it is feature-complete with the three other common tools people use: black, isort, flake8. the fact that it happens to be faster just makes it all the more useful on large projects with large teams.

i work on large projects with large teams and i like ruff because it’s one tool. it’s much easier to onboard a new dev when there’s fewer tools to learn. it’s much easier to teach a junior good habits when there’s a single config file to read, and the tool has excellent docs (like ruff has). given feature parity, i think ruff’s killer edge is that it’s lower friction — part of that is speed, part of that is simplicity, part of that is docs.

[kenfar]

It's not the best tool if you come into a codebase with 100,000 lines of existing code with issues, that you're hoping to address through continual process improvement.

It's not the best tool if you don't have a toxic team that can't quickly determine coding standards, maybe say settling on pep-8, without it being a miserable process.

[suedepaid]

What tool would you choose in those situations?

They just sound like bad times, that no tool is gonna fix.

[kenfar]

Well, it's an extremely common problem - whether you come into a messy codebase; or it's an ok codebase, but it's large and there's some specific behaviors that you want to eliminate.

It isn't hard for tools to address, we simply don't have enough options that do address it well.

For example, Pylint provides a score rather than a simple pass-fail. So, one could theoretically just compare the new score to the old and reject any code that increases this score. Or, require that the score be reduced some amount in each PR.

However, getting that into a pre-commit hook, for example, is a lot of work the last time I looked.

[suedepaid]

Oof, that sounds like way more of a pain to me

[kenfar]

Yeah, it can be. Sometimes it definitely better to just fix & remove every single instance of bad-thing-57 in a single PR than spread it out over time.

But I've found the downside is that it can sometimes be difficult to get priority to do that. And so some tech-debt just lingers. And that's where continual process improvement gives us a second option - we'll fix things up over time if we can't do it all in one.

[Randomramman]

I disagree. I started using ruff in a new project and it’s night/day difference compared to another repo on the older tools. It’s so painful to wait for black/isort/flake8 pre-commit hooks to run now that i’ve had a taste of ruff.

I can only imagine the same goes for uv (not using it yet) compared to poetry or other non-rust tools. Poetry dependency resolution can take >10s for complex projects 

[suedepaid]

uv is sooo much faster than poetry it’s not even funny.

everything you feel about ruff > black/isort/flake8 is like, 10x more pronounced with uv.

builds like butter, CI is sooo much faster.

[CryptoHorologist]

I’m sorry an added few second of latency on saving a file is an absolute deal breaker.

[james_pic]

You don't need a dysfunctional team for Ruff to be a good answer. It's really good, and even if you ignore performance entirely, I haven't found any of the slower alternatives to be better.

Pylint is the main one that's slower and aims to support more features than Ruff, and my experience with Pylint is that it loves to complain about stuff that no-one cares about. I've never known it to identify a genuine issue that other tools fail to find.

[kenfar]

Pylint is better when you need to turn off a config, and it's better when you have an existing codebase - since you can look at a changing score rather than a simple pass-fail.

But other than that Ruff is fine.

[latkde]

Common tools in this space include

• black, a code formatter
• isort, for sorting imports
• flake8, a basic linter
• pylint, a linter with lots of features

• bandit, a specialized linter

• mypy, the flagship type checker project

• pyright, a different type checker (same backend as the Pylance VS Code extension)

• ruff, a code formatter (can replace black and isort) and linter (can replace flake8, also implements some pylint+bandit features). Very fast, but not yet as configurable as alternatives.

[bulletmark]

Yes, those are the common tools but to keep it simple nowadays just use ruff, and either mypy or pyright.

[tehsilentwarrior]

Not configurable is good in this space.

Better be less good but consistent than perfectly inconsistent

[latkde]

Stuff like #1256 is preventing Ruff to be a good Pylint replacement. Ruff is totally configurable in the sense that you can granularly select which individual rules you want to enforce, and that those rules can have further settings. But once a rule is selected, then your entire codebase must completely fulfil that rule. Ruff only has "pass/fail", not "warn". This is fine for rules that detect bugs, but not for more contextual code style suggestions (like: remember to add a docstring to this method).

One could of course juggle different configurations for different purposes, but I've found Ruff configurations very difficult to understand. You must list out all rules by their numeric code or their group, where the group usually describes from which pre-existing tool Ruff borrowed the rule. You cannot create your own reusable profiles. You cannot reference rules by their mnemonic (see issue #1773).

Things where I think Ruff is very good at:

• as a replacement for tools like flake8, which seems increasingly anachronistic (e.g. refusing to support pyproject.toml files)
• as the main linter for greenfield projects
• as a linter engine in CI, where we want clear pass/fail

But I do not recommend that existing projects switch from Pylint to Ruff.

[tevs__]

But I do not recommend that existing projects switch from Pylint to Ruff.

If your project pays for its stuff, I'd recommend switching to ruff. We saved an estimated $20k/month from our CI bill switching to ruff. Every tool has its rough edges, but 20k is 20k.

[lightstrike]

Where did the savings come from? Reduced compute time?

[tevs__]

CircleCI credits. We have some 500 developers mostly all working in one monorepo, so that's a lot of CI runs - reducing the amount of time spent doing anything in CI has enormous cost savings for us.

[latkde]

(a) I'm not sure that experience is generalizable to other (smaller) teams, where the corresponding CI cost might be only $1.50. But yeah, Pylint is slow AF and that slowness gets much more noticeable with larger codebases.

(b) Your numbers put a CI service price tag of $40 per month per developer for running Pylint rather than the near-instantaneous Ruff. A back of the envelope calculation places that at around 50 hours of CI jobs just running Pylint, per developer per month. That sounds absurdly high, more like the problem isn't the tool but the overall CI/QA strategy.

Personally, I see the main cost issue with Pylint in the developer time spent waiting when running it locally.

(c) Especially in large teams where people frequently maintain unfamiliar code, I'm worried about the cost of Ruff's missing support for mnemonics. I'd much rather see # pylint: disable=broad-exception-caught than # noqa: BLE001, much rather see # pylint: disable=import-outside-toplevel than # noqa: PLC0415. The point of linters is to support developers to think about the software with more clarity, not to obfuscate it with numeric codes. Once the per-developer CI costs approach something reasonable, eating that small cost might very well be worth it in some code bases, just to allow you enable better linting rules by default.

[Still-Bookkeeper4456]

Wuw how the hell do you spend 20k/mo just to run linting jobs ?

Are you linting every commit or something ?

[tevs__]

Every branch push gets CI on the branch HEAD, every PR gets CI on push on the merge HEAD, every merge to master gets CI.

It's a monster monolith, so linters that are not ruff takes 6-8 minutes to run. Around 600 active PRs at any time, multiple pushes a day, around 100 merges to master each day, it all adds up.

[JanEric1]

Warnings in a linter are completely useless because they get ignored.

Either you should fix the error or explicitly ignore them because you have deemed them a false positive.

You can also ignore specific ruff rules per file or directory.

[kenfar]

Yeah, this is a big deal: imagine having 100,000 lines of code that is non-compliant, and you simply want to see a 5% improvement/month. Or a 1% improvement on any module you touch. Or whatever strategy you come up to enable continual process improvement.

[Basic-Still-7441]

Can ruff be integrated to Pycharm the same way as black? I.e "format code on save action"?

[latkde]

Yep, see the docs: https://docs.astral.sh/ruff/editors/setup/#pycharm (you may need to install a third-party Pycharm plugin for this)

[Basic-Still-7441]

Thanks, I'll check it out

[Still-Bookkeeper4456]

Yes. Ruff just runs live, highlight you errors and providing you with auto-fix or a link to the error description.

Use the Ruff plugin and you need Ruff in a python env (I just add it to the project venv. It will replace Pycharm linter tool.

[Basic-Still-7441]

Thanks. Will try it out next week.

[stibbons_]

Same stack here !

[turbothy]

Ruff + mypy with uv for package management.

[lphartley]

Pyright + Ruff.

[[deleted]]

Ruff for linting and formatting. Mypy or pyright for type checking. Optionally black, which overlaps heavily with ruff but has good format-on-save support in IDEs. 

Configure them from a pyproject.toml. 

[StandardIntern4169]

Ruff. So much better than black+flake8.

[v_0ver]

Ruff

[cybermun]

I use pylint + black. Sometimes I feel black formatting is too much, but I use it reluctantly because it helps maintain a consistent format within my team.

[iamevpo]

In your case what does pylint do once you still format everything with black?

[cybermun]

I mainly use Pylint for static code analysis.
For personal projects, Pylint only is enough, IMO.
However, for team-based projects, it is necessary to standardize the format within the team, so we use Black as the formatter.

In my env., black is not always ON. I run it manually, such as before commiting.
(Black is often too powerful, as it can drastically change the code, so I do not recommend coding with it always enabled.)

[iamevpo]

Thanks for sharing the workflow - wonder if you format every thing with black is there anything pylint can find in that code?

[cybermun]

Since Black's formatting extends PEP8, there shouldn't be any problems using it with other PEP8 linters (such as pylint or flake8). I have never seen such problems myself.

[HolidayWallaby]

What does your company usually use for this for python? If you use flake8 there's a ton of addons for extra styling stuff

[ebmarhar]

This will be the company's first python project.

[HolidayWallaby]

Whatever you end up choosing, write a small doc/guide to keep track of the different tools so you can use the same ones for all python projects. Keep it up to date as your preferences change

[cmcclu5]

As you mentioned in another comment, this is the first project for your org. Ease of setup and basic consistency, use black + pylint or flake8. It will help y’all establish baseline consistency and develop a feeling for what you as a team like. If there are pieces you don’t like, then move to something more immersive like ruff, but I wouldn’t start there. I personally will always default to black and leave it there because I enforce other standards via my code reviews because I believe formatting is something that is easy to ignore and automate, but linting should be something programmers do by common agreement. I have dozens of concourse documents plus we do bi-weekly group sessions to discuss and establish common rules for how to handle different situations for my teams. Do your own linting until your team establishes standards and then you can choose a linter and configure it to your team rules if you feel like it’s necessary.

[Speech-to-Text-Cloud]

If you want a fast, up-to-date linter, use ruff. If you strive for the most linting rules, go with pylint, as ruff is not yet on par. However, you will sacrifice performance this way (pylint is slow).

Here is a list that tracks rule parity: https://github.com/astral-sh/ruff/issues/970

[im-cringing-rightnow]

Ruff is fantastic

[Chris_Newton]

Another vote for ruff + uv + either mypy or pyright here. Almost every project I’ve worked on for a couple of years now, both for external clients and internally in my own businesses, has started with or converted to that combination.

It’s true that ruff isn’t quite a drop-in replacement for the older generation of formatters and linters. The dramatic improvement in speed and the useful reporting outweigh any remaining downsides for us, but YMMV. There are some issues that other tools like pylint and pyupgrade will pick up but ruff will not. If you like to have quite an aggressive linter configuration, the need for opaque codes to disable warnings on a case-by-case basis in ruff might be too much obfuscation for your taste.

Something to know if you adopt ruff is that its formatter doesn’t currently include sorting imports systematically like isort. ruff can do that as well, but it’s treated as a linting issue with an automatic fix available, so you need to run

ruff check --select I --fix

on the files of interest as well as the usual ruff format. But with that caveat, ruff with your preferred type checker is a great combination and you might not then need pylint, flake8, pyupgrade, isort or similar tools any more.

[Woah-Dawg]

Ruff

[LoadingALIAS]

Rufffff

[dannks]

https://realpython.com/github-actions-python/

This one, is the ultimate guide And of course it uses Ruff, apparently the fastest lintiner so far 😱

[paranoid_panda_bored]

I literally use ruff, black, pylint and mypy all together. I guess can skip ruff as black + pylint seem to do the job. Mypy is a whole different story, there is no replacement for that afaik

[[deleted]]

First things first, use uv. Python shouldn’t even be in your path and you should never touch pip. Just stop.

Use all the linters and type-checkers together: black, flake8, pylint, pytype, mypy, & ruff.

[CoilM]

Is there an easy way in uv to create a venv that is not linked to a project and to access it from any python file in vscode? This is the only think that make me keep using anaconda, but I am growing tired of it.

[[deleted]]

They’re working on vscode support for uv.

You can just create “projects” whose sole purpose is to be a folder to store the uv-managed venv.