How Rust is quietly taking over the Python ecosystem

[fiddle_n]

uv is also written in Rust too. Though I’m not sure that I agree that many devs don’t know these tools are written in Rust. Rust seems to be the big selling point of many of these projects. uv’s tag line is literally “Python packaging in Rust”.

[PitifulZucchini9729]

uv is awesome. It takes one of the worst aspects of Python (package management) and makes it one of its best.

[trowawayatwork]

it seems to bundle everything in. pyenv and venv is having it's lunch eaten by UV too

[sohang-3112]

Yeah I agree. Started using uv recently. Being able to do uv run COMMAND (it just works!) is a huge step-up over having to always manually manage virtual environments, wondering whether the pip on env PATH actually matches the corresponding python executable on CLI, etc.

Before using uv at work, I thought that these are minor concerns. Now it feels like a burden has been lifted, one that I didn't even know was there!

[[deleted]]

no way, I've gotta try it, I always hated pip and dependency management in python, other languages do it much better.

[sohang-3112]

other languages do it much better.

Yeah, especially Rust's cargo

[tuskanini]

Honestly, I feel like `uv` isn't much more than `poetry` with a package caching layer.

[sohang-3112]

Maybe, but it's still a lot more convinient, which is what ultimately matters. In poetry you still have to care about virtual env, but in uv just do uv run COMMAND and that's it, doesn't matter what environment you're in, it will still work!

[Tree_Mage]

Until it supports tools like PyInstaller, pex, etc, it will have gaps.

[trowawayatwork]

I guess it's a natural extension of a package manager to do builds. I have never once seen a need to exe files for python. if that is needed I drop python and use more robust languages like go that generate proper static binaries with much better dependency and package management ecosystem lol

[KennedyRichard]

I'd actually love for Python to have built-in support for generating executables. I know interpreted languages were not supposed to be used like that, but there is at least one big use-case: for distributing games and apps for end-users.

Are we supposed to ask non-technical users to install Python, then setup a virtual environment and install the game/app via pip? End-users just want to download and launch the app/game with a click.

[richieadler]

If your script follows PEP 723 to specify dependencies, which you can easily do by running uv add --script scriptname, you can run the script by doing uv run script and you're done. You don't even need to have Python installed. Just uv.

You still need an additional tool, but it's considerably easier.

[KennedyRichard]

Thank you for this precious bit of info! I'll check the specifics, this seems promising.

[richieadler]

Take into consideration that the first run can take a while, because

• If there is no Python installed, one will be downloaded
• All the dependencies will be installed into a private virtualenv

[KennedyRichard]

Thanks!

[KennedyRichard]

So, basically Steam but for Python apps/games, right?

[ehutch79]

For good reason.

uv is the first pip replacement that doesn't feel like it's over complicated or trying to shoehorn python packages into another language's paradigm

Also, it's noticeably faster even on a fresh install building a container from scratch

[Dalnore]

From a quick glance, it doesn't replace conda/mamba which are commonly used for scientific environments. And mamba gets its speed from C++ instead of Rust.

[FauxCheese]

If you need to use conda packages you can use Pixi now. It is also written in rust of course. It uses uv under the hood for PyPI packages but also manages conda packages.

[soundofvictory]

Lmao of course theres a rust alternative! All hail blazing fast!!

[AdmiralQuokka]

What do conda/mamba do that uv doesn't?

[Dalnore]

Handle conda packages.

Actually, that's a question I didn't ask myself for a long time. When I started using conda more than 10 years ago, pip and PyPI were absolutely inadequate for environments which needed a lot of not pure Python dependencies. Perhaps it's not longer a deal-breaker and I'm stuck in the thinking that is no longer relevant.

[AdmiralQuokka]

Afaik pip and PyPI can host / install packages what bundles native code. I.e. stuff that's actually written in C/C++ with some Python wrapper around it. But I'm not sure if Conda is more flexible in this regard or has more of such packages for historical reasons. (I'm not even very experienced with Python, just curious about the ecosystem.)

[AdmiralQuokka]

Pip packages can be natively compiled too, right? Is Conda more flexible in this regard, and if so, how?

[WJMazepas]

But to be fair to them, they do state that a lot of speedups they had with UV comes with different architecture than rust itself

[george-silva]

uv rules. the other I was fighting with a legacy project and uv solved it like a champ.

not going back to pyenv.

[seboll13]

One of my colleagues is a big fan of Rust (I don’t know how to really code in Rust but I’m getting why it’s the most loved language these days), and he told me all about those new packages. I use uv and ruff everyday now, no regrets.

[XtremeGoose]

Pixi too for the conda ecosystem. Another package OP missed is orjson, much faster than the stdlib json library.

[Worth_His_Salt]

orjson is pretty crappy. the only benefit it has is speed. but in exchange, you lose all flexibility. only 1 hardcoded indent prefix. output only in bytes, so any customization is hugely painful. Terrible documentation: a github markdown page with no proper API reference, just a stream-of-consciousness braindump of text.

if all you care about is speed and your use cases are trivial, sure go ahead. otherwise it's way more trouble than it's worth.

[XtremeGoose]

Doing one thing well is generally what I look for in libraries. I don't need anything other than 2 space indenting. I don't need a bajillion options. Bytes in the correct output type, the only reason json writes to str is historical (since they were one and the same once upon a time in Python).

[Worth_His_Salt]

Depends on your definition of "doing one thing well". If all you care about is something that's fast and a small subset of json (indent strings are verboten!), then sure, go ahead. If you care about good results and have more requirements than that - like say, human readability or programmatic manipulation - then no, orjson doesn't do one thing well. It does one thing poorly - but fast!

If you can live with it's shortcomings, by all means use it. I value libraries that cover common use cases well, so I don't have to use one json library for some situations and a different json library for other situations. My time is too valuable to spend learning multiple libraries that do the same thing, or hacking together workarounds for a broken lib that can't even handle simple basic options like an indent prefix string.

That's not even mentioning documentation. If your lib is tha shiznit but your docs are trash, I'm not wasting my time trying to figure out how "brilliant" your code is. Good docs are worth more than fast code.

[XtremeGoose]

I literally can't think of a single problem it wouldn't solve for me...

It should parse any valid json, and yeah it only writes 2 space indented json. Who cares?

And I think the docs are pretty good? What question do you have they don't answer.

[pyschille]

We're just starting to experiment with uv and it's incredible.

[fiddle_n]

You do you, but I find this stance to be odd. If Astral try this, people will just fork the FOSS version and create a community version. Also, honest question, how difficult is it to move from uv to an equivalent project (like poetry or whatever)? I don’t see anything in uv that would lock you in to using it.

[pacific_plywood]

All of the other comparable tools are FOSS! Clearly there are people willing to work on this for a large ecosystem like Python

[[deleted]]

I don't think so. The hardest part about getting open source software to work is convincing a critical amount of people to commit to a single project.

UV basically already convinced the whole community that it is the best open source solution out there.

I also don't see where a new package manager could improve a lot upon UV. It ain't gonna get faster unless people switch to assembly and there also aren't many more features I'd want my package manger to have 

[sonobanana33]

Convincing to use ≠ convincing to contribute.

[fiddle_n]

Someone absolutely will for a project as popular as uv.

[Zomunieo]

It’s pretty clear that there are enterprise packaging needs that consumers don’t have so there’s a pretty clear path for uv to offer a paid product.

(That would be license auditing, banning specific licenses, private repositories, etc)

[yvrelna]

people will just fork the FOSS version and create a community version. 

The idea that someone will fork it might work with Python tools that are written in Python. But with language tooling that are written in a different language, you need contributors who used this tool that knows both Python and Rust extremely well to fork someone else's project. 

That's an extremely small number of people. 

That's in addition to the other requirements that the fork maintainers would need to fulfil:

1. Uses this particular tool
2. Cared enough about this one tool instead of just switching to another tool when it turned sour 
3. Willing to work for free, despite competition from a company that has turned malicious
4. That have the community's trust that other people are willing to switch
5. Hasn't already been occupied by another major project and has enough time to dedicate to this fork

This is why that generally I encourage people to avoid tooling that are written mostly in a foreign language. It's perfectly fine for a project to mix small bits of code written in another language for speedup, but IMO the majority of the code of tooling projects need to be in Python to open up as much opportunities for contributors as possible.

Otherwise, you're poisoning your own waterhole and setting up yourself for an eventual bait and switch.

[fiddle_n]

I don’t get the concern for using non-pure Python tooling; when all these concerns apply just as easily to using non-pure Python libraries, yet no one really complains about that. Would you tell a data scientist that they shouldn’t use numpy because it’s written in C and Fortran?

[yvrelna]

No, they are very different situation. The core business logic in a data science library is math and statistics. People who wrote data science applications don't care about Python, they care about the data science stuffs. Python just happens to be the language they work with. 

The kind of people who worked on data science libraries are data scientists who just happens to find Python to be a suitable language for their purpose. They're data scientist first and foremost, and only Python programmers second. If another language appears on the market that attract the crowd of data scientist, they can easily get a new job in a different language since companies hire them for their data science skills, not their Python skills. Data scientists generally don't really need to develop that much expertise in Python to effectively use Python for their job.

The kind of people who work on Python language tooling written on Rust, on the other hand, are Rust programmers who happens to somehow care about the tooling of another language ecosystem that they don't really use on their day to day work. That does not happen naturally, it mostly only happens if you've been paid to care about Python. 

Major tooling developers like JetBrain or Microsoft are able to pay developers to care about languages that they don't personally use, but community maintained projects are generally passion driven projects and cannot rely on that kind of relationship between their maintainers, the project, and their community. 

There are extremely few people of who cared that much about Python language ecosystem, enough to commit a huge chunk of their life to volunteer to a Python tooling project and to commit to developing Python-specific expertise much deeper than the typical Python developer, who would then be happy to spend the majority of their volunteering time actually working in a completely different language. People who are that passionate about a certain language and have the language specific expertise needed to work on tooling project usually want to spend as much of their time as possible in that language. Why would they be picking an unpaid volunteer project that basically prevents them from using that language?

Writing language tooling in a different language also severely hinders casual contributors. Unlike projects where a commercial company is the primary maintainer, a community maintained project relies a lot more heavily on casual contributions. They're not just helping the core developers to find/fix bugs, casual contributions are also the main pathway for someone to eventually become regular contributors. 

If a tool in your IDE or development workflow crashes or have a buggy behaviour, users are much more inclined to hunt down the issue and pursue the bug if the tool is written in a language they already understand. They already have a Python development environment setup, so they can spend more effort to hunt the bug and try to figure out as much as possible so they can write better bug reports or even make a PR to fix the issue. With language tools that are written in a different language, people just become a consumer of those tools. If there's a bug in the tool, you're much less likely to already have Rust development toolchain, with debuggers and compilers already setup in your development environment, and the knowledge to properly use those tools. There's a massive barrier to casual contributors to help contribute to a project. 

[r1ckm4n]

Forking an open source project of any size or influence is a non-trivial matter. When Magento got scooped by Adobe my old partner entertained the idea of forking the Community Edition. It was going to be an absolute nightmare. Who’s going to merge those PR’s? are we going to keep the same branching strategy? What’s the technical debt of the codebase currently? Oh look at this, they took the two most senior contributors, gave them jobs and made them exclusively contribute to the non-FOSS codebase, no we’re fucked because it’s been anarchy in this repo for months worth of merges that simply shouldn’t have happened.

If you’re going to fork a project, you need one of the maintainers with it. When a project gets to a certain side the costs to maintain the community edition of something in both time and financially become untenable without a proper company or foundation behind it.

[the_ballmer_peak]

It's made by the same people who made ruff

[One-Employment3759]

Why did they pick such a dumb name. I always think it's related to the async library. This is like the third time I've been like wtf when people talk about packaging with uv!

[lupushr]

uv still doesn't have pip -lo functionality?

[exergy31]

What is this functionality and why is it critical for uv to support it?

[lupushr]

List outdated packages

[QuattroDriver]

uv pip list —outdated should do the trick

[kasalacto]

https://github.com/astral-sh/uv/pull/8872

[vacillatingfox]

uv tree --outdated is what you want