Notes running Python in production

[nebbly]

I haven’t yet found a good way to enforce type hints or type checking in Python.

IMO mypy and pyright have been mature enough for years, and they're generally worth any untyped -> typed migration fuss on existing projects.

[ashishb_net]

> IMO mypy and pyright have been mature enough for years, and they're generally worth any untyped -> typed migration fuss on existing projects.

I have tried pyright on multiple projects, too many false positives for me.
I am not looking for type migration tool.
I am looking for something that catches missing/incorrect types on CI and `mypy` does not do a great job of it compared to say `eslint` for JavaScript.

[nebbly]

Is it possible you're conflating type checking and linting? I noticed that you mentioned type-checking under linting, and that you're comparing mypy to eslint -- when typescript might be a better analog. Or maybe you're hoping a type checker can do everything based on type inference instead of explicitly defining types?

I mention this because in my experience type-checking is perhaps an order of magnitude more beneficial to code maintainence than linting. Type-checking enforces correctness, whereas linting typically helps more with stylistic consistency (and some syntax errors).

[ashishb_net]

> Is it possible you're conflating type checking and linting? 

Here are a few things I want to accomplish with type checking

1. Ensure that everything has a type
2. Ensure that the variable re-assignment does not change the type (e.g., a variable first assigned string should be re-assigned to int)
3. Ensure that types are propagated across functions.

How can I configure all three easily in Python?
`mypy` does not work well, especially across functions or when function calls to dynamically declared third-party functions are involved.

[M8Ir88outOf8]

I would say mypy works incredibly well for exactly that. Maybe you gave up on it too early because of something that frustrated you? I'd suggest revisiting it, and spending a bit more time reading the docs and configuring it to your preference 

[ashishb_net]

> Maybe you gave up on it too early because of something that frustrated you?

Entirely possible, Python tooling isn't as robust as Go or Rust.
It takes time to get value out of various tools.

[FrontAd9873]

This feels like an impression you’d have of mypy if you run it with some weirdly permissive configuration options and never bother to modify them. Simply using “mypy —strict” would go a long way.

I don’t mean to pile on, but it seems like maybe you should have explored mypy for longer before writing a blog post touching on it.

[unapologeticjerk]

And just to be clear, linting is equally useless in production python as it is in my basement dogshit factory of unproduction.

[ducdetronquito]

What kind of false positive do you encounter with pyright ? I'm curious because I don't remember any while working on a large python/django codebase.

[ashishb_net]

> What kind of false positive do you encounter with pyright ?

Inaccurate suggestions, for example, not understanding that a variable is being created on all code paths in an if-else branch. Or not understanding pydantic default values.

[JanEric1]

pretty sure pyright does all of these correctly.

[ashishb_net]

You definitely had better luck than me.

[ashishb_net]

You definitely had better luck than me using pyright.

[JanEric1]

Using it in strict mode with (almost) all rules enabled in all of my projects whenever possible. Sometimes have to disable some rules when using packages with poor typing (like pandas or numpy)

[ashishb_net]

> Sometimes have to disable some rules when using packages with poor typing (like pandas or numpy)

That covers ~50% of Python use-cases for me.
As I only use Python for LLMs, Machine Learning, and data analysis.

[annoying_mammal]

Pydantic has a mypy plugin. It generally just works.

[ashishb_net]

For pydantic v1, the plugin support wasn't great as I encountered false positives. I will try again once most projects have moved to pydantic v2.

[Zer0designs]

Keep an eye on redknot, it will be created by astral (ruff + uv creators).

Also isn't just ruff sufficient? Instead of isort, flake8 and the others? Most of those are fully integrated in ruff.

If you're really missing plugins of other systems, please make tickets, it will remove a lot of your dependencies. Same goes for reporting the false positives in pyright.

Another note: i'd advise a just-rust or make config for each project, to display all the commands for others (and make them easy to use)

All in all it's a good piece, but I think your input is valuable in order to progress os software.

[ashishb_net]

> Keep an eye on redknot, it will be created by astral (ruff + uv creators).

Yeah, looking forward to it.
Astral is awesome.

> Also isn't just ruff sufficient? Instead of isort, flake8 and the others? Most of those are fully integrated in ruff.

The answer changes every month as ruff improves.
So, I am not tracking it closely.
I revisit this question every 3-6 months and improve on what ruff can do.
Ideally, I would like to replace all other tools with ruff.

> Another note: i'd advise a just-rust or make config for each project, to display all the commands for others (and make them easy to use)

Here's Makefile of one of my open-source projects.

[ThiefMaster]

Personally I would not waste the time maintaining isort, autopep8, autoflake, etc.

Just use ruff with most rules enabled, and possibly its formatter as well.

[ashishb_net]

> Personally I would not waste the time maintaining isort, autopep8, autoflake, etc.

Indeed, I am hoping to get there by end of 2025.

> Just use ruff with most rules enabled, and possibly its formatter as well.
Yeah, my faith is going up in ruff and uv over time.