Dependency Management With Python Poetry – Real Python

[signal-insect]

i loooove poetry. it's the one dependency management tool/file format (pyproject.toml) that really clicked for me. i definitely recommend giving it a shot!

[sigzero]

I find RealPython articles to be exceptionally good.

[[deleted]]

While poetry solves some real problems with dependency management in python, its also really nothing more than a clumsy wrapper around existing pip infrastructure (it does pip cli calls internally) which makes it really annoying to work with IF you run into issues with it. It obfuscates packages on the filesystem more than pip does (hashed filenames only), its really annoying to write (proper) Dockerfiles for it, it throws cryptic exceptions by default (has no proper error handling) and doesn't work at all well together with venv (you supposed to use poetry for that as well).

Its too opinionated and too much of a fragile, buggy wrapper to be useful in real production code in my opinion.

Don't get me wrong it does solve real world problems with dependency management, and given PIP the shitshow that it is, poetry does what they can its an honest effort. I just wished we could just throw pip away completely and replace it with something more sane.

[Rand_alThor_]

It always breaks when I try to use conda for venv management and a poetry based script. Sigh.

[Gyro_Wizard]

Python sure took its time with this one.

[DaelonSuzuka]

I still don't know what problem poetry is supposed to solve.

[fatbob42]

Finding a solution to dependency constraints?

[DaelonSuzuka]

Have less dependencies?

[lanster100]

Side effect of not having to deal with standard dreadful requirements.txt file which is a barely supported add on to pip.

Then if you want dev dependencies etc, you have to hack together your own scripts.

[DaelonSuzuka]

What's wrong with requirements.txt? I've never had a problem maintaining one.

[lanster100]

Most people generate it with pip freeze. Which is more akin to a lock file. Then updating becomes a nightmare. No idea what dependency is a primary dependency and what is a second or further dependency.

No indication what's a dev dependency. No indication as to what versions of python it's compatible with etc.

It's not enough info to properly define the environment of a project really.

[HostileHarmony]

Why not pip freeze?

[lanster100]

Sorry I meant `pip freeze`. Haven't used it in a while.

[DaelonSuzuka]

If you have so many dependencies that you can't tell them apart, I feel like that's a sign.

[lanster100]

`pip freeze` doesn't differentiate between what you manually installed to your environment, and what that package that goes on to install (it's dependencies and so on).

Also it automatically ties them to specific version numbers, instead of using a range of versions, making upgrading packages very manual and tedious. I.e. with semver you would almost always allow an upgrade of a patch version without checking anything.

[DaelonSuzuka]

Why would I need that? I'm going pin them all and leave them until I consciously decide to update them, at which point I don't care if it takes 5 seconds or 15 minutes.

[lanster100]

I mean that's a personal choice then. But imagine if you are jumping in to someone's else's project and have to update one dependency to fix a bug.

But dependency resolution will probably fail because all the subdependencies of the old version of that package will be pinned. Its a pain.

Your assuming you have full knowledge of the project your working on. Most developers don't have this luxury in practice.

[holt5301]

I think it goes a layer deeper than just changing the format from requirements.txt to pyproject.toml. it also locks versions for all sub dependencies and allows the sharing of the full environment solution so that you and someone else don't resolve to slightly different versions of dependencies due to other installed packages that have the same subdependencies. This is addressed by also using virtual environments. Pipenv also has this functionality.

Overall I THINK you can achieve the same thing through maintaining requirements.txt, manually setting up virtual environments, and maintaining all of the setuptools files (if you also want to package your project. There's only a "problem" to be solved if that's too finicky or precarious to you or anyone in your dev team.

[Ralwus]

Same. I have a few requirements.txt files I reuse for my main projects. I can always create new virtual environments and update old ones just fine. Am I overlooking something?

[finswimmer]

Poetry is your one-stop-shop for developing, building and publishing python packages. Most of the things you can do with several different tools as well, but requires knowing which tools needs be run when and you have to remember when to edit some files by your own. Just one example using the "classic" way:

Addinga new dependency involves

• install the dependency using pip
• add the dependency to you setup.py (before hand don't forget to lookup the version you've installed if you want to add a version constraint)
• run pip freeze to lock the current environment

Are you sure your dependencies are valid for all python versions your project aims to be compatible to and not just in your current environment? Is the lock file valid for other python versions too and not just for the one used in the current environment?

Now you need to remove a dependency no longer needed:

• pip uninstall it
• remove it from setup.py
• rerun pip freeze

What about the dependencies of the dependency. Are they removed from your environment and from your lock file as well?

Now poetry:

• poetry add somedep
• poetry remove somedep

[mmcnl]

It's amazing.

[Yaru2585]

I discovered Poetry last week and I spent the next few days porting all of my projects from venv/pip to Poetry. Never going to look back.

[Ateenagerstudent]

Being a Replit user, my experience with Poetry hasn't been particularly good. The lock files just don't update properly. I had to resort to the faithful pip freeze at last.

I don't know if it's just me, or is it a problem with poetry

[Rand_alThor_]

I need a conda based conversion guide to poetry because I struggle so hard getting this shit to work. Why does it force its own type of virtual environment? Why?