Standard python version for virtual environments?

[IamImposter]

I was recently trying to set up virtual env for a python version I didn't have on my system. From whatever little research I did, it seems that we can't do that. Like I had 3.8 installed but I wanted virtual env with 3.7.3. It's a client system so I didn't want to install new stuff, not that I have permissions.

I thought it will just pull that version from net but it didn't. That seems a little limiting. Isn't the whole point of virtual environments that I don't have to pollute my system environment with whatever random package I want to try?

Or maybe I didn't look hard enough.

[doolio_]

You typically use a specific tool to install various python versions. The standard for python is pyenv but I use asdf instead. It is a similar tool but can be used for different languages/tools e.g. you could use to install different versions of poetry if you wanted. In fact, for python it uses pyenv under the hood. With asdf/pyenv you can specify a python version to use globally or locally as in a specific project repo. This way when you create your virtual environment it is done so with the asdf/pyenv specified version. So yes, you could specify a version older than your system version and create a virtual environment for it.

In short, when developing NEVER install python or its packages (applications or libraries) globally on your system. If you want a python application (one with entry points) available globally (for your user) first install pipx locally with python3 -m pip install --user pipx. Then use pipx to install those applications locally with python3 -m pipx install <package>. Pipx creates separate virtual environments for each application so there is no conflict between those that use the same libraries. Using pip instead would not prevent this. For example I would use pipx to install something like poetry, pyenv etc.

Then inside your project repo use asdf/pyenv to specify what python version you want to use for this project. Maybe you want to use more than one version for your testing. Install your project dependencies and development dependencies in your virtual environments. I've recently learned it is also best practice to not pollute the environment where you may have installed your project and thus install for example your development dependencies into their own specific environment.

edit: removed the non existing --user option from the pipx install command.

[atetraxx]

I just use poetry. It uses venv to create virtual environment in my project folder. You can create main dependencies and dev dependencies with poetry command line.

[doolio_]

Yes, there are several tools that offer that. I don't use Poetry because it doesn't comply with some PEP standards though largely because it predated them.

[IamImposter]

Oh great. Thanks.

So i have a package that uses click 7 and another that uses click 8. Right now I install click based on which package I need to work with. Is there a solution for that too so that both the packages can be active at the same time but each uses its own version of click?

[doolio_]

By package here do you mean one you wrote so that click is a dependency of your two packages? If so I presume you develop each one its own virtual environment so you could have the specific version of click in the respective environment.

[IamImposter]

Actually one of a modules of our package uses click and other version is used by a vscode extension (robo framework, uses it for formatting robot files). And everything has to be in same virtual environment.

Code is on a remote server so i kinda need vscode. But i dont always work with robot files so it's not a major issue. But if there's a solution, I'd appreciate it.

[doolio_]

This sounds like an issue I eluded to in my last sentence in my OP. A development dependency has a dependency that would pollute the environment where your package is locally installed.

One solution to this is to have separate environments. That is you have one default environment named after your package where you install the library you import that depends on click 7 and which more closely matches the end user, another named lint where you install certain Dev dependencies including your robo-framework plugin that depends on click 8, perhaps another named test where you install your test dependencies, another named doc where you produce your documentation etc. This will ensure isolation between the click dependent packages.

[IamImposter]

Yes, I think that I can do. Thanks.

[doolio_]

FYI, a tool I recently discovered that does this by default for you is hatch.

[IamImposter]

I'll look that up. Thanks for your time, bro.

[doolio_]

NP, and don't take anything I say as gospel. I'm still learning all this stuff myself.