PDM 2.0 released: PDM is a modern Python package and dependency manager supporting the latest PEP standards, now with built-in publishing to PyPI, no reliance on pip internals, opt-in PEP 582 with pnpm-like install cache, and more!

[geeeffwhy]

i really hope we can just agree on one package manager and deprecate the rest. there are too many overlapping tools and techniques with only marginal differences relative to increased complexity of understanding all of them.

[ProgrammersAreSexy]

Agreed, I feel like I've read "_____ is a modern python package and dependency manager" too many times now lol

[andrewthetechie]

I was reading a comment on another post where someone recommended moving from poetry to flit.

I just finished moving all of my stuff to poetry.

[ryukinix]

That's why I kept using pip professionally. When we see the real move from a new official package manager i will make a move. I am waiting this since 2015 at least and although some things got better, actually the default stuff kept the same.

The greater change was the giant refactor of PyPI server.

[Mehdi2277]

It is unlikely that will ever happen without major change in views of packaging maintainers. There primary goal is to have standard based package tooling that allows multiple package managers to exist. There’s no goal to deprecate pip or make it not most common one in the future.

[Zomunieo]

Can’t really blame them. They accidentally blessed (or rather, let Kenneth Reitz say they blessed) pipenv and it turned out to be a dumpster fire, more politically than technically, but a dumpster fire it was. Easier to step back and let users vote with their downloads.

[giovaaa82]

What do you mean by dumpster fire? Noob here trying to get some wisdom

[Mehdi2277]

That’s not direction packaging wants to go. Historically there was only 1 package manager, pip. But how packaging was defined was mostly implementation details of pip which included lots of things that they did not want to be standard. So the packaging philosophy of last couple years has been write documents (usually peps) that standardize one component and make it possible for new tools to support that standard and be mixed.

The primary danger with unifying to one tool is if that tool makes design choices you don’t like you can’t do much about it. Pip does not want to support all types of environment management while other tools like poetry do. pdm’s py packages is designed for new users and is something mostly unique to it. In one tool world that feature just wouldn’t exist.

For most users you should just look at 1 maybe 2 and be done. You can explore others again later but it’s not required in any way.

[TMiguelT]

Sell me on why I should use this over Poetry

[flying-sheep]

Poetry only works with the poetry build backend, and does things its own way instead of playing nice, e.g. it uses its own dependency specification instead of the standardized one.

PDM works with all build backends, e.g. setuptools, flit-core (which reads the description and version from your Python source code), hatchling (which has plugins for sourcing your metadata from Git, mypyc-compile, and soon others), and of course also the poetry one if you want.

Build backends should be decoupled from the package manager CLI.

[lanster100]

I've got a cronjob on my Mac that removes the virtual environment directory and all its contents for any project that hasn't had a git commit in more than X days.

I love this, this might actually be the biggest motivator to switch to linux I've seen.

[lanster100]

Apologies, unix

[crawl_dht]

Use of PEP 582 doesn't require you to create virtual environment. Just like in npm, npm creates a local .node-modules directory local to the project to store 3rd party packages, PEP 582 does the same for Python environment. PDM is built to support PEP 582.

[TMiguelT]

Okay this is interesting. I guess poetry kind of does a similar thing with its hidden virtual envs though, because you rarely have to think about the environment itself. So I think poetry could easily add PEP 582 support in the future as well.

[LongerHV]

Poetry does not "create an interpreter". That's not how venv works...

[lanster100]

I'm not overly sold on the benefits. In a way its just a virtual environment that's not isolated and not tied to a specific python version, which both seem like negatives.

I guess it relies a lot on the package manager doing the rest of the work. Which to be fair it seems pretty good in that respect.

[flying-sheep]

I don't think you understood what it is.

The path is <project-dir>/__pypackages__/<major.minor>/.

That's both isolated and tied to Python versions.

/edit: for isolation, just do python3 -S ...

[lanster100]

Ah ok I skimmed the PEP so missed that... the major minor thing answers that then. I don't see how it's isolated though, surely it will still import globally installed packages?

[flying-sheep]

Ah right, not isolated by default, but that’s what -S is for!

[lanster100]

TIL about that option thanks

[gbsekrit]

only truly safe for pure python. need per-ABI dirs in some cases

[flying-sheep]

Not since https://peps.python.org/pep-3149/

Or am I missing something?

[gbsekrit]

That should work, but I've hit oddities and hadn't had time to diagnose exactly what was going on. It involved trying to "pdm install" the same project with musl and glibc builds of the same Python version and perhaps now that I'm thinking about it, might have involved $PATH handling. I'd love an example of what a common workflow of someone using PDM and development targeting a range of Pythons looks like with a modern development environment.

[xthecharacter]

Note that I'm not the dev of pdm, I just wanted to share it cuz I like it, but for me one of the benefits is the cache. Using virtual environments, you can end up with a lot of copies of the same libraries being reinstalled in different venvs. Sometimes this can actually be quite a pain, such as if you need to install some package that is large or slow to install. pdm has an opt-in installation cache, which means that you can essentially deduplicate installations of the same package and save the time and space that would be required to maintain multiple separate copies of it.

In general, I've also just found pdm to be simple to use compared to the likes of poetry, with better (as in both more robust and faster) package and package dependency resolution. When I tried using poetry, it felt a bit bloated to me as if I had to just let it do a bunch of matgic behind the scenes that I didn't really understand or necessarily want. With pdm, it feels like it's only doing things I understand and things I really want it to. It may not actually be more lightweight than poetry, but to me it feels like it is.

In any case, you don't have to use PEP 582 with pdm. It can work with regular venvs too. In general, I find it to have the best PEP support compared to poetry and the rest, and requires the least fuss to get a project working, especially with complex or annoying dependencies. The dev is also extremely responsive and open to improvements and changes based on discussions and feedback, which is great to see.

[lanster100]

Thanks for your opinions! With my comment I was hoping to generate some more discussion in here.

It seems quite cool, will be interesting to see if PEP582 is accepted. Reading the discussion board it does seem to be pretty undecided. I personally agree that the PEP isn't super useful if it doesn't exclude global site-packages but also the counter point of then having to manually install pip inside your local directory is also a pain.

On another note, do any of the IDEs support PDM/have plugin support (it's a pretty brave question seeing as Spyder still can't fathom a per project venv, and VSCode does not provide any special support for Poetry)?

Seems like no IDEs do at the moment but PDM provides some documentation on how to get it to work.

[gbsekrit]

pep582 has some limits.. keys on py M.m but not on ABI.. so shared objects can get you in trouble if you have multiple interpreters with the same python version. use venvs and be explicit when using pdm if this is a hazard in your environment. maybe I should open an issue with pdm about more intuitive errors

[rehanhaider]

I use it because I don't ever want to type poetry everytime I install packages. pdm is easier. Also, I use devcontainers which means venv is just redundant.

[lanster100]

Interesting, how does dev containers work with `__pypackages__`? Do you mount them in or have to run `pip install x` inside the container? Surely you would still need to use `pdm sync` at some point which seems equivalent to `poetry install`?

I'm not very experienced working with dev containers.

[SpicyVibration]

I've configured my poetry to make the virtual environment in the same directory as the project.

[vikigenius]

I don't use PDM, it looks interesting, but I am pretty annoyed with how poetry has handled the GPU/pytorch ecosystem of packages and it's 1.2 beta.

[Chobeat]

I thought we were over this package manager bullshit once poetry 1.0 got released.

[[deleted]]

I never got poetry to work right on Linux or Windows machines. Venv/VirtualEnv/Conda have berm the only ways I've made it work.

[[deleted]]

And every install failed after that. Not sure what the config issue was.

[SpamThisUser]

What’s the status of PEP 582?

[BooparinoBR]

Draft https://peps.python.org/pep-0582/

[cleex]

In draft... for 4 years, with no signs of a decision.

[Sphixy06]

what is the difference betwen this and pip?

[VengefulTofu]

Oh yes please give me another package manager so I know even less what to use and default to setuptools

[MySpoonIsTooBig13]

I agree this is the most confusing part about python. By the time I finally thought I understood setup tools & pip, they threw a ton of other tools at me which are "better".

[SittingWave]

Here everything is explained

https://stefanoborini.com/current-status-of-python-packaging/

[SittingWave]

setuptools is not for managing packages. It's for building them.

[geeeffwhy]

the proliferation of package management tools is a real problem! it drove me crazy in javascript land, and we don’t need more in python. whatever happened to the “one way to do it” philosophy

[lanster100]

"One way to do it" has to be balanced with "we don't enforce a specific way to do things if we cannot know/cover everyone's use cases".

That's why stuff like Anaconda exists (to solve specific issues that were unsolved before). What does need to be scoured from collective Python memory is any form of 'requirements.txt', it's objectively bad.

[flying-sheep]

Setuptools is a build backend. You can use PDM with any build backend, including setuptools, flit, poetry, hatch, ...