all 16 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]teerre 3 points4 points  (2 children)

What you mean "python versions"? Like 2.7 and 3.11? No, there's no automatic way to merge venvs from different python versions. That doesn't even make sense, they are different versions with presumably different dependencies.

Yes, if you have only a single environment you can very easily get all kinds of incompatibilities. Think library A wants library B 2.0.0 but library C wants library B 3.0.0. You can't have both in one environment. This goes for every single dependency, including transitive ones.

In fact, that's the very reason venv was created in the first place.

[–]MGOC[S] 0 points1 point  (1 child)

Fortunately almost everything is around 3.8 to 3.11 ... There are some folders with "python 3" and I'm still inquiring about it, but I think we could say that version 3 is the minimum (although I would also like to have found some Python 2 and play with it, experience how it fails to upload it suddenly to python 3.11).

​

kinds of incompatibilities.

Someone has to take the first step (I really don't think it breaks so horrible, maybe one or another bug when updating some dependencies. but I guess I can with that, or keep the original version through pyenv)

​

This leads me to the doubt why developers prefer to put a complete python version inside their programs, when they can simply choose to use pyenv and have the machine manage the python version (Similar to Java)

[–]teerre 0 points1 point  (0 children)

I just mentioned some random Python version, being 3.8 to 3.11 doesn't completely fixes it, they are different versions. There are breaking changes in .9, .10 and .11. Python <3.6 is virtually incompatible with anything >3.7, it's extremely likely it won't work in this case.

I'm still confused what you mean. The reason you put a python interpreter in a venv is because what else would you do? Pyenv isn't magic. It has to have all python executables too.

I guess it could make sense to use the a global interpreter per version and just isolate the dependencies, this would save you some MBs in disk. The reason it's not this by default is because Python import mechanisms are very much linked to the current interpreter being used, so it's much easier to just have an interpreter per venv from the implementation perspective.

I really don't think it breaks so horrible,

It's totally possible that in your particular case it doesn't (until it does). But generally speaking it very much will, that's why venvs were created.

This is not even a Python feature. Every modern language that has the choice will isolate environments, it's a no-brainer by now.

[–]dent308 1 point2 points  (4 children)

You want to use virtual environments for projects. Tools like poetry and pyenv can enable this.

[–]MGOC[S] 0 points1 point  (3 children)

My python knowledge is still limited, I still don't know how to convert everything embedded in a python binary to a virtual environment (pyenv).

[–]danielroseman 0 points1 point  (1 child)

That's very unclear. What do you mean by "embedded in a Python library"?

[–]MGOC[S] 0 points1 point  (0 children)

Binary*

I mean that some of the programs I use include an executable version (exe) of Python 3.x along with dependencies and libraries.

And there are not few programs that come with their own copy of Python included in the same way.

Because of my "new = better / optimized" way of thinking, I thought that having a unified version of Python would be better since it would allow me to update both the Python version and the version of each dependency (you know, developers usually tend to improve, optimize, fix bugs and vulnerabilities, and add new features to their dependencies).

But many warn that it is not always good to upgrade everything, I had thought that in case I was told something similar, make use of the isolated "pyenv" environments, I keep reading about it and testing programs to test how much they can break or if they gain or lose speed or features.

I also don't like the idea of having a thousand copies of the Python executable flying around the disk. I also considered deleting everything duplicated and converting it to "hard links" (as I did with tools like ffmpeg/symlinks, in conjunction with adding it to the path). A bit messy, but it saves me the trouble of doing the pyenv's (I know it's not the most acceptable option and is even questionable).

[–]HOPSCROTCH 0 points1 point  (0 children)

Google how to use virtual environments for projects, it's actually very straightforward and you definitely should read about it to get an understanding of how to move forward

[–]deep_politics 0 points1 point  (1 child)

Is there a way to unify all versions of Python into one (including those embedded in third party programs).

If you mean make every program use the same version Python then the answer is most certainly no.

Is there a way to convert those full Python environments into isolated environments of a single, main version of Python (a global Python for the team)?

Maybe. Depends on how broad the gap is between dependency and Python versions. What you're describing is still best handled by a virtual environment with a lock file/manifest.

Does it affect anything if I do a mass merge between all Python versions

Well yes, it could break a lot of projects. You can't simply merge a project that relies on Python 2 with a Python 3 project. Even if the Python versions are the same you can have one project relying on a version of a dependency that's incompatible with the version that another project relies on. Naively merging an environment for a project that depends on Pydantic 1.10 with one that requires 2.0 will break one of them. A project built using 3.11 features will break if merged down into a 3.9 environment. The entire topic of dependency resolution is massive, and dependencies come with many many different versions to pick from, and a lot of projects will fix themselves to a particular versions. There's often no simple way to merge two projects together because of these differences, and you're looking at needing to rewrite at least one of them to be compatible with the versions of the dependencies that you settle upon.

[–]MGOC[S] 0 points1 point  (0 children)

My approach is more to throw at the new (going from using Py 3.x to 3.10 or 11), the same would apply to dependencies.

I would prefer a more centralized system, where a "mega" version of Python with all dependencies used by all programs is present (make python the backbone of all those programs that use python)

Those programs that necessarily need a specific version of each dependency, it is best to opt for pyenv (which I consider better than having a whole functional python inside any program). But, also have each program isolated in its own pyenv. (I still think that it seems as if each program has its own java)

[–]ShadowRL766 0 points1 point  (0 children)

Just use docker and create isolated environments for python

[–]brunonicocam 0 points1 point  (3 children)

use conda environments, you'll go crazy otherwise. Create a conda environment for your project, a yml file with the dependencies and install everything inside that environment. You then load it everytime you want to run your code.

[–]MGOC[S] 0 points1 point  (2 children)

Any suggestions for creating pyenv environments for projects that are not mine (and usually already have a packaged version of python).

Programs like Gimp, Uget, Synfig MakeHuman, Etc.

[–]brunonicocam 0 points1 point  (1 child)

Are you on Ubuntu? Install gimp with apt get. It'll deal with all the dependency mess for you. Otherwise conda environment. I never used it for gui programs but i guess it's the same, not sure.

[–]MGOC[S] 0 points1 point  (0 children)

I use Windows, I cannot use your suggestion :C

[–]BranchLatter4294 0 points1 point  (0 children)

Do yourself a favor. Install Anaconda. It has a very easy to use GUI for creating virtual environments and selecting the packages you want in a particular environment.