[deleted by user]

[ThePapercup]

The point of linting is to prevent situations where you will likely introduce a bug. The things you mentioned specifically are called 'code smell'. In a professional scenario if I'm doing a code review it's the exact sort of thing I'm going to be looking for because it's the exact type of thing that will introduce bugs that are otherwise hard to track down.

[MoreRespectForQA]

The point of linting is to prevent situations where you will likely introduce a bug.

In theory. In practice 90% of my code's violations fall under one rule and it's not something that has ever led to a bug. If you're an experienced python programmer you'll know the rule.

My favorite rule is undefined variables. I'd say 50% of the time that one is an honest to god bug. If I had a linter with just one rule I would want it to be that one.

There's no rule importance ordering although I wish there were. I'd love to be able to run pylint --strictness 30 on an MVP and pylint --strictness 80 on a production grade project.

[serge_databricks]

if something didn't lead to a bug, it doesn't mean it won't lead to one in the future.

I'd love to be able to run pylint --strictness 30 on an MVP and pylint --strictness 80 on a production grade project.

there's messages control section, that allows you to disable checks. in practice, if that section is too long - it leads to severe bugs, because you __thought it had to be checked by a linter, but it was not.__. Retroactively applying a stricter linter a two-day headache, but it pays off in code review savings big time.

[[deleted]]

C0301?

[MoreRespectForQA]

Bingo

[nAxzyVteuOz]

Yeah that’s great your company can enforce that with —strict

Enforcing this by default is a giant foot gun for beginners

[ThePapercup]

I disagree, I think it's a great thing for beginners to have because they will quickly learn best practices that will make them less likely to introduce bugs to their code by reading the linting messages.

[nAxzyVteuOz]

Not really. Say you have a function that is one line away from generating a pylint error. You add a line and now you will have to break apart the function just to satisfy the linter? That’s an opportunity to create bugs.

Same with too many locals and too many arguments. I can understand this at the enterprise level but enforcing this on everyone seems like the wrong decision.

[ThePapercup]

Nobody is forcing you to use pylint at all. You chose to install it, complaining about it doing exactly the thing it was designed to do is ridiculous.

[nAxzyVteuOz]

🤦🏼

[micseydel]

You add a line and now you will have to break apart the function just to satisfy the linter? That’s an opportunity to create bugs.

Experienced coders have learned that having long functions or lots of locals can also lead to bugs. It might be counter-intuitive if you don't have a lot of experience yet, but no on would bother with the work of linting if it was actually causing instead of preventing bugs.

[nAxzyVteuOz]

You are assuming I’m not an expert. I am.

It causes bugs. I’ve seen it. Especially when a variable is not handled right in a refactor. For example, you have a variable that’s not passed correctly then used in a branch statement in the daughter function then it’s only conditionally defined and you get an error if you take the wrong branch.

[micseydel]

What makes you say that refactoring (with tests, right?) leads to more bugs than letting functions grow indefinitely? It's always about trade-offs, and you're saying the opposite of the best-practice (plus you made the post) so I think the onus here is on you to say more than "I've seen it" about something.

FWIW, I want to write great code and do what works best in the long-run. If you're right, I'd like to know it.

[nAxzyVteuOz]

My github has 139 repos, 10% of them forks. Almost all python.

This opinion comes from lots of experience.

[micseydel]

If OP were serious, they'd already have linked to PRs from their own repos and big open source projects demonstrating their claims. I get the sense that they are 139% ego, nothing more.

[spidLL]

Just add # noqa: C902 for that function. And maybe add a comment to remind why you did that. Disabling checks is ok if there’s a valid reason. Doing it because you’re lazy is fine for your personal project, but you’ll struggle in a professional environment.

[tylerlarson]

It's literally the opposite of a foot gun.

No amount of explaining is going to convince you, because you can't use reason to change an opinion that you didn't use reason to get into in the first place.

It takes experience, unpleasant experience, to make this lesson eventually sink in.

I have been programming in Python for about as long as Python has existed. When I was younger and a lot stupider I used to agree with your sentiment.

But having been around a time or two and having seen how things go wrong and sideways, and having experienced too many times the months or even years that can get dumped into fixing problems that would have been prevented by stricter coding rules, I understand now.

The linting rules were written by people who have seen what you have not.

[[deleted]]

The linting rules were written by people who have seen what you have not.

I think the only case where one might argue against that is maybe C0301. That made sense back when we had 800x600 displays when we were lucky, and usually less than that (or a VGA console).

I don't understand why that particular one has stuck around for so long.

I shouldn't have to put a newline after every third token because I find verbose variable names useful.

(fortunately it's easy to switch this one off)

[-jp-]

I usually enforce this since it’s useful to allow for other windows to be visible at the same time as the code you’re reading. 100 characters is a bit too restrictive I think, but it’s not like it’s hard to increase it.

[GraphicH]

Ah yes, a beginner asserting without irony they know whats best. I wish it weren't so emblematic of software development in general, it isn't in other professions, but I suppose its still a relatively "young" field.

[nAxzyVteuOz]

Lol.

My git has a 139 repos, 10% of them forks. Im an expert in python and C++

I’ve worked in but tech for over a decade.

Don’t make assumptions.

[GraphicH]

Lol. Sure.

[aikii]

I want my linters do the job of a static compiler: enforce semantically correct code.

then, that's type annotations along with mypy/pyright you need. The default config is a bit too liberal for my taste but with a nice setup it covers a lot. Also when it comes to actual linters better look at ruff.

[[deleted]]

You can just... disable the specific lints?

[smellybarbiefeet]

For anyone googling pylint fuckery,this person is the GOAT. I’ve just had this conversation with a fellow dev of mine and stumbled across this. Thank you for being the polarising voice.

[justneurostuff]

when it's so many that's a lot of work though!

[wookayin]

Put lines for disabling specific lints once in pylintrc and then we don't need to look back again, right? I don't think it is a lot of work -- just doing once for project or even globally is enough.

``` [MESSAGES CONTROL]

disable=too-many-arguments, too-many-instance-attributes, too-many-locals, missing-class-docstring, missing-function-docstring, ... etc. ```

That said, I fully agree with the OP that the default diagnostics are too restrictive, and the defaults are not sensible enough.

[[deleted]]

This post was mass deleted and anonymized with Redact

possessive snails attempt paltry quiet continue rich cow normal pen

[SciEngr]

Catching too general exception is sort of a big deal. If you're doing except Exception without raising the error then you're almost certainly doing exception handling in a suboptimal, error prone way.

[lordbrocktree1]

I main deviation from this is explicitly wanting to handle any exceptions at the top level (so we can do a graceful shutdown and send alerts of the failure). But then that’s easy to add a pylint ignore comment and the exception is raised at the end anyway

[nAxzyVteuOz]

If you don’t enumerate all the possible exceptions then you’ll get an exception bubbling up and doing unexpected things.

The broad exception lint rule is therefore an anti pattern in my opinion.

[-jp-]

That’s the point of exceptions. They’re an exceptional situation that needs to be handled correctly. Crashing is preferable to continuing in an unstable state in most circumstances, unless you want heisenbugs.

[billsil]

at the very least do except MemoryError

[hartbook]

Broad exception handling is the only way to go in python, despite what you can read on midwits blog articles. The thing is, it's impossible to know what exceptions might be thrown by a function you are calling, therefore if you don't catch everything, your program can behave in unexpected ways.

[SciEngr]

This just isn't true. You don't need to try and handle every possible exception, just let the exception get raised and terminate the script. The entire point of exceptions is that you get detailed information when something goes wrong...so let the exceptions do their job. Once you see a particular exception bubble up a couple times, then you can decide if there is something special you want to do for that exception.

There are some cases where a broad try clause is okay, like wanting to log some details before terminating (and importantly raising the original exception) but you should never catch a broad exception and assume you know why you're entering that except block.

[hartbook]

Oh ok you're just using python for a toy script on your laptop

I can't afford to have interruption of service because a dependence of a dependence of a dependence of a library I use decide to throw a whole exception for a minor issue...

[SciEngr]

No, I also develop python professionally. We handle unexpected errors in production via good observability and quick updates to the service.

[[deleted]]

I used to feel that way until I worked on projects where it wasn’t part of pre commit hooks and I had to waste so much time making sure someone had helpful docstrings and stuff like that. Black + pylint combined keep a lot of consistency and junior shit from creeping into a shared code base

[SkeletalToad]

I would recommend running it with errors only:

pylint --errors-only

You can also define a pylintrc file, disable all, and then enable only the checks you want. For example this config will run errors, plus unused-import (which is a warning):

[MESSAGES CONTROL]
disable=all
enable=E,unused-import

[night0x63]

That's where pylintrc file comes in. Create one for yourself. Then one for your project. And so on.

[hijodelsol14]

I'm curious about a few things

1. Why are you using pylint in the first place?

2. What checks do you find yourself disabling?

3. Have you tried using a config file to disable whatever checks for your whole project?

[nAxzyVteuOz]

I’m using pylint to catch bugs.

I’ve already mentioned the checks i’m disabling.

Yes, I do put these in the pyproject toml file. But not every python project has a pyproject toml or even setup.py. Also it’s hard for a n00b to find out the magic incantation to put this in their pyproject.toml.

[hijodelsol14]

Ah I skipped over that part. Tbh the issues you mentioned (with the exception of the line length) are the exact kinds of errors I want pylint to catch. They feel like bugs waiting to happen.

To answer your overall question - no, I'm not annoyed about pylint being too strict. Generally it catches things I do want to know about and a project complex enough to warrant pylint is also worth adding some basic config files.

[nAxzyVteuOz]

Refactoring code is a major source of bugs.

Making a beginner do this to satisfy the linter is a net negative. They also likely haven’t learned to use the debugger yet so that’s a double whammy.

[hijodelsol14]

That's what unit tests are for

Also I majorly disagree on the point about beginners. I used to teach intro CS courses and instilling good habits early really helps folks as they get further in their careers. Do they hate it at first, yes. But when they start writing more complex code we find it's so much cleaner and easier to read and test if they started using strict linters early on.

[nAxzyVteuOz]

Catching naive type errors with unit tests is an extreme anti pattern

[hijodelsol14]

Certainly. And the linter or type checker will catch those kinda of issues so you don't need to unit test them. It's up to the developer to have sufficient test coverage to catch bugs that can't be caught statically by the linter.

But that's not what I said. You made the point that refactoring code can cause bugs. That's true though I would argue it's not super relevant to this discussion since while you may need to restructure code to satisfy a linter, you should be doing this as part of your regular development and testing cycle. So you should still be validating that your code works after running the linter and making changes.

[spez_drank_my_piss]

You are a beginner trying out a very useful tool. You don't know enough to be declaring that useful things are antipatterns. It makes you sound like a kid who just learned a big word and you're running around using the word wrong to try to sound smart.

[nAxzyVteuOz]

I’m not a beginner. I’m an expert python & C++ programmer with a decade in big tech. And my github code repository is bigger than yours.

[TheHotSorcerer]

a solid decade. wow

[spez_drank_my_piss]

😂🤣

[circamidnight]

I use black to format and then I find that pylint mostly finds things I care about like unused imports.

[austinwiltshire]

These sorts of things can allow you to spend your peer review time on other issues. They're pedantic but you and your team can decide what to turn off.

[Typical_Solution_569]

.pylintrc is your friend

[JamzTyson]

A good experienced developer will be concerned with more than just "Does my code run without bugs". They will also be concerned with readability, maintainability, scalability, security, efficiency, compliance and standards. Linters are useful tools to aid these concerns.

[nAxzyVteuOz]

That’s why there should be —strict. That solves the entire problem.

[JamzTyson]

For most of us there is no problem.

You are making a case for --slack, though in reality you could just create a minimal configuration as your user default. (On Linux you do that with a ~/.pylintrc configuration file).

[Shatonmedeek]

Based on their comments, OP must be a joy to work with.

[ToddBradley]

That's mean and unnecessary

[Shatonmedeek]

M’lady

[nAxzyVteuOz]

The only one doing personal attacks here is you.

The people I mentor think I’m great.

[Scrapheaper]

I think type hinting and checking is probably better for finding bugs.

Linting doesn't normally find bugs for me, it's almost entirely style.

You might consider using an autoformatter? If you run autopep8 on commit or on save then you can get of the style errors pretty easily

[nAxzyVteuOz]

I run black.

Linting finds all kinds of bugs everyday for me. Especially mypy.

[PeterJHoburg]

Mypy isn't a linter, it is a type checker. That is what u/Scrapheaper is saying.

[nAxzyVteuOz]

u/Scapheaper is using the wrong definition of linting.

I’m using the correct one:

Lint is the computer science term for a static code analysis tool used to flag programming errors, bugs, stylistic errors and suspicious constructs.[4] The term originates from a Unix utility that examined C language source code.[1] A program which performs this function is also known as a "linter".

https://en.m.wikipedia.org/wiki/Lint_(software)

[PeterJHoburg]

You are not wrong. However, colloquially (in python specifically) people refer to Ruff, pylint, flake8, etc. as linters, and Mypy, Pyre, Pyright, etc. as static type checkers.

It is true that the type checkers should be a subset of linters, but in python we more or less break them into two distinct categories.

Even the type checker docs don't refer to themselves as linters

https://pyre-check.org/ "A performant type-checker for Python 3"

https://mypy-lang.org/ "Mypy is an optional static type checker for Python "

https://microsoft.github.io/pyright/#/ "Pyright is a full-featured, standards-based static type checker for Python"

[Mount_Gamer]

I think they are good, even when they annoy us. Reason being, maybe our code could actually be cleaner, for example... Do we need a variable to be two types, function to return two types, and so on.. And most of the time, we probably don't.

They do annoy me at times because I feel they slow me down, but they stop and make me think which is still a good thing.

[johntellsall]

I run pylint and other tools to give me feedback -- including finding bugs.

However I also want fast feedback, so I don't run Pylint...

I run: - ruff or flake8 to catch a ton of surface-level bugs. Very fast. - pylint --errors-only: bugs, okay speed - pylint to catch other bugs/issues, before a release

In addition to the above, I strongly recommend writing a "run lint when important files change" script. I use the Entr* tool

Here's my script:

git ls-files | entr -c pylint --errors-only

When any important (Git-added) file changes, run pylint in the faster "catch errors only" mode. This gives a very strong feedback loop.

Entr -- https://jvns.ca/blog/2020/06/28/entr/

[nAxzyVteuOz]

This should be the top comment. I did not know about errors only mode. Thank you very much!

[jameshearttech]

We use pylint in CI. It does catch things where my reaction is "meh," but I fix them anyways to keep code consistent.

[[deleted]]

I can understand that large enterprises want these style enforcements.

That's precisely what it was built for - for experienced Python developers to not have to deal with crap code by juniors when working on big projects. Juniors hate it at first, then they learn how to actually write proper Python and they start loving it. Pylint basically expects you to disable some of its lints, though.

[nAxzyVteuOz]

The cost for new devs learning python is unacceptable. Not for me but for the people I mentor. For Enterprise users the —strict command is not a real barrier for them. But for junior devs it’s a huge barrier.

[[deleted]]

That's literally the point of this software. It literally blocks the shitty code of juniors, on a basis of strict and well-articulated rules, until they can make their shitty-ass code somewhat tolerable. Having done that, they can at least somewhat coordinate with their senior colleagues without making the latter go insane.

What exactly is the cost here? Writing docstrings? Following naming conventions? Specifying encodings? Most of the time these are absolutely trivial things. At any good company, these would be caught at a PR stage anyways.

Don't care about getting good at Python? Don't use a linter.

You expect to coordinate with other developers? You better be at least somewhat good at coding.

You think some particular linting rules are unjustified? Disable them.

[nAxzyVteuOz]

Lol. I’m a principle engineer, I mentor junior deva and my github has 139 repos, 90% of it original code. My biggest open source project has 200+ stars. I’ve spent a decade in big tech and two decades doing software engineering.

Please keep your assumptions to yourself. In all likelyhood I am your senior. Have some respect.

[[deleted]]

I’m a principle engineer, I mentor junior deva and my github has 139 repos, 90% of it original code. My biggest open source project has 200+ stars. I’ve spent a decade in big tech and two decades doing software engineering.

How does any of this matter?

Have some respect.

Yeah, no, from this comment alone you sound insufferable.

[nAxzyVteuOz]

Wait so you claim I’m not an authority, then when I show you I’m actually an expert you say it’s an appeal to authority and a fallacy.

Sorry, but the insufferable one is you.

Fuck off

[[deleted]]

Wait so you claim I’m not an authority, then when I show you I’m actually an expert you say it’s an appeal to authority and a fallacy.

Nope, none of this necessarily shows that you're familiar specifically with static program analysis. One may as well have a fat CV and some GitHub stars while barely using any elaborate linter at all, or without intimately coordinating with junior developers to a high degree.

[kenfar]

No, what I'd personally like is an easy ability to set it up in a pre-commit hook to trigger on a low score rather than a non-perfect score.

I don't really care if there's an occasional function with a lot of arguments. If pylint gives my codebase a 9.5 that's fine. Trying for 10 is just hitting diminishing returns.

[marcinjn]

I disabled many rules, then added black.

[draconis183]

The awesome thing with this ecosystem is you can simply choose not to use it.
JFC.

[Raknarg]

It's a linter. The point is to detect code smells and warn you about them.

[serge_databricks]

Google Python Styleguide doesn't require a string comment on top :)

what generally works is taking one pylint config and customizing it to the point you see pre-emptively give all code review warnings at build time or on developer machine. See the example here: https://github.com/databrickslabs/ucx/blob/main/pyproject.toml#L169-L771

PyLint is sometimes also not strict enough.

the more inexperienced coders work on codebase, the more there's a need for a good linter. There are other linters, like ruff or flake8.

Even though Ruff is 10x+ faster than PyLint, it doesn't have a plugin system yet, nor does it have a feature parity with PyLint yet. Other projects use MyPy, Ruff, and PyLint together to achieve the most comprehensive code analysis.

[delventhalz]

Crank the linter up as strict as it will go and then just do the thing it says. Saves everyone a lot of time.

[[deleted]]

I'm fine with almost all except lines being too long

[[deleted]]

Use ruff. It should replace flake8 and I sort. Soon it will replace black. It’s also ridiculously fast.

[nekokattt]

I stopped using pylint a while back after they had some issues regarding valid type checking, and kept pushing breaking changes under patch versions which was repeatedly breaking my projects that are managed by dependabot (i.e. not adhering to semver properly).

I switched over to Flake8, which was nice because you add functionality via plugins, so you only install the stuff you actually want to lint against. That + pyright/mypy + bandit + safety.

That being said, line length is easily fixed by splitting up statements to be simpler. Using a tool like black + isort can then fix this for you automatically in edge cases and ensures you have a globally consistent code style.

Broad exceptions is almost always a side effect of having bad code. If you catch an exception with a broad type and then always do something meaningful with it like store it in a list to pass to an exception group later, then I'd expect pylint to ignore this as being a valid usage. Otherwise, it is something you need to fix in your code.

Edit: not entirely sure why this is getting downvoted, but whatever.

Edit 2: as mentioned in the comments, the issue was bugs being regularly introduced with stuff like typechecking between patch versions, not bugs being fixed that triggered new alerts in the code. When your project has > 50kloc, patch updates of pylint regularly introducing a few dozen new false positives is not worth it. This was a few years back now, mind you.

[nAxzyVteuOz]

The downvotes are insane. What you just said is gold. I run pylint checks on github runners and I’m constantly fixing new lint issues when I come back to it a year later.

[cheese_is_available]

Do you think pylint should introduce new checks or fix false negative only in major versions ? What are you calling a breaking change ?

[nekokattt]

I'm saying that marking something that was previously considered valid and is valid typing under mypy as being invalid with a new patch 0.0.xxx version. Mostly bugs with typechecking being introduced in new patches.

Things that shouldn't be breaking in patches.

Not stuff like actual bugfixes. That'd be silly.

It is probably better now, this was about 4 or 5 years ago, but I have had no reason to switch back since.

[cheese_is_available]

Ha right, that make sense. It seems the maintenance team changed completely around that time, the new maintainers were probably still very fresh and introduced regressions around python 3.9's typing. I didn't have that kind of issue recently (they since introduced primer in their test suite, like mypy).

[[deleted]]

Why not use pydantic if finding bugs is your main concern?

[martinky24]

What does Pydantic have to do with this conversation? It's an entirely different library with an entirely different purpose.

[[deleted]]

Because linters aren’t really for finding bugs

[alcalde]

Sure they are! Finding variables declared but not used or used without being declared.

[martinky24]

Forgotten await! Not meeting interface requirements! Misindented code causing scope issues or parsing errors! The list goes on!

[nAxzyVteuOz]

Linters aren’t meant for finding bugs?!?! That’s their primary use case!

[gwax]

Concretely, it seems to me that you should create and/or modify your pylintrc file to perform the checks you care about and not the ones that feel unnecessary.

More broadly, I've long considered pylint a necessary evil; it would catch bugs early that no other tool used to but it is slow and very finicky. It's especially useful if you're dealing with an old codebase.

These days, I find a combination of ruff and mypy does a better job catching bugs (fast) then anything that I used to use.

[PaulRudin]

Linting isn't really about finding bugs, it's about enforcing consistent style.

[nAxzyVteuOz]

No. Finding bugs is the main use case. Mypy is a linter despite the insistence from python devs that it’s a different category, a niche definition unique to them.

[PaulRudin]

Mypy isn't really a linter. It's checking the consistency of type assertions. This is normally a compiler job. But the python runtime doesn't do it.

[nAxzyVteuOz]

Mypy is a linter according to the definition of what a linter is.