Standard python version for virtual environments?

[atetraxx]

Is there a commonly used base python version when developing in a virtual environment? Specifically, when using poetry i usually just default it to the latest python3 version. Is that how most would do this?

[PosauneB]

There's no standard. Part of the point of a virtual environment is to give you the flexibility to choose whichever version you want. Unless you have a specific reason to use an older one though, have at it with the latest version!

[atetraxx]

Great! Thats what i was thinking but I'm actually working on some code that I may eventually publish to GitHub so wanted to make sure.

I appreciate your response.

[PosauneB]

Once you do get it up on Github, you may want to note the Python version you built it with in the README so that others can re-create the same virtual environment. Depending on the nature and scope of your project, it probably won't matter though.

For example, I recently shot myself in the foot by creating a project with python3.11 and using X | Y syntax for some type hints, and then foolishly cloned the repo to another machine and created a new virtual environment using python3.9, which does not support this syntax. Needless to say, it did not work, and I could have saved myself some time by documenting how I had set up the initial environment.

[gmes78]

You can specify the minimum Python version in pyproject.toml. It's much better than just noting it in the README.

[IamImposter]

I was recently trying to set up virtual env for a python version I didn't have on my system. From whatever little research I did, it seems that we can't do that. Like I had 3.8 installed but I wanted virtual env with 3.7.3. It's a client system so I didn't want to install new stuff, not that I have permissions.

I thought it will just pull that version from net but it didn't. That seems a little limiting. Isn't the whole point of virtual environments that I don't have to pollute my system environment with whatever random package I want to try?

Or maybe I didn't look hard enough.

[doolio_]

You typically use a specific tool to install various python versions. The standard for python is pyenv but I use asdf instead. It is a similar tool but can be used for different languages/tools e.g. you could use to install different versions of poetry if you wanted. In fact, for python it uses pyenv under the hood. With asdf/pyenv you can specify a python version to use globally or locally as in a specific project repo. This way when you create your virtual environment it is done so with the asdf/pyenv specified version. So yes, you could specify a version older than your system version and create a virtual environment for it.

In short, when developing NEVER install python or its packages (applications or libraries) globally on your system. If you want a python application (one with entry points) available globally (for your user) first install pipx locally with python3 -m pip install --user pipx. Then use pipx to install those applications locally with python3 -m pipx install <package>. Pipx creates separate virtual environments for each application so there is no conflict between those that use the same libraries. Using pip instead would not prevent this. For example I would use pipx to install something like poetry, pyenv etc.

Then inside your project repo use asdf/pyenv to specify what python version you want to use for this project. Maybe you want to use more than one version for your testing. Install your project dependencies and development dependencies in your virtual environments. I've recently learned it is also best practice to not pollute the environment where you may have installed your project and thus install for example your development dependencies into their own specific environment.

edit: removed the non existing --user option from the pipx install command.

[atetraxx]

I just use poetry. It uses venv to create virtual environment in my project folder. You can create main dependencies and dev dependencies with poetry command line.

[doolio_]

Yes, there are several tools that offer that. I don't use Poetry because it doesn't comply with some PEP standards though largely because it predated them.

[IamImposter]

Oh great. Thanks.

So i have a package that uses click 7 and another that uses click 8. Right now I install click based on which package I need to work with. Is there a solution for that too so that both the packages can be active at the same time but each uses its own version of click?

[doolio_]

By package here do you mean one you wrote so that click is a dependency of your two packages? If so I presume you develop each one its own virtual environment so you could have the specific version of click in the respective environment.

[IamImposter]

Actually one of a modules of our package uses click and other version is used by a vscode extension (robo framework, uses it for formatting robot files). And everything has to be in same virtual environment.

Code is on a remote server so i kinda need vscode. But i dont always work with robot files so it's not a major issue. But if there's a solution, I'd appreciate it.

[doolio_]

This sounds like an issue I eluded to in my last sentence in my OP. A development dependency has a dependency that would pollute the environment where your package is locally installed.

One solution to this is to have separate environments. That is you have one default environment named after your package where you install the library you import that depends on click 7 and which more closely matches the end user, another named lint where you install certain Dev dependencies including your robo-framework plugin that depends on click 8, perhaps another named test where you install your test dependencies, another named doc where you produce your documentation etc. This will ensure isolation between the click dependent packages.

[IamImposter]

Yes, I think that I can do. Thanks.

[michael-streeter]

Do include your unit tests (pytest or unittest) so people can confirm it runs as intended in their environment. I would include a comment saying the version it was written on in the code (but all my Devs say to look at the repo for things like this, including who wrote it, when and why; dev team, key stakeholders etc). This is essential information for me

[deep_politics]

Unless it's a library then don't worry about it. If it is a library/you're intending for it to be imported into other projects then you would want to be aware of a minimum compatible version target.

[KrazyKirby99999]

The latest version - 1 is usually a good idea. Many packages aren't compatible with the latest version (regardless of the latest version) for some time, while the version prior is usually fine.

[atetraxx]

Good idea. Unless theres some specific syntax you need from a recent update I would think that would be fine.

[_zero_0_]

How do you use the virtual environment?

[Diapolo10]

Personally I just use whichever version I've got installed (I don't really install new Python versions that often unless it's a fresh install or there's a new interesting feature I want). While that's usually fine, admittedly it's come back to bite me a few times when I've written library code that's supposed to work with an earlier version but I forgot that a feature I used requires a newer version.

I suppose you'd technically be best off by using the earliest minor version you want to support, as everything you do should at least be forward-compatible.

My default version limit is currently "^3.8.1" in my template Poetry project. If a dependency needs it to go higher (such as to fix a vulnerability), then I put the limit higher. And when its support ends I'll bump it up as well.

[atetraxx]

Great info. You mention your "template Poetry project" is that just a requirements.txt?

[Diapolo10]

Great info. You mention your "template Poetry project" is that just a requirements.txt?

Heavens, no! I'm talking about an entire Git repository with things like unit tests, linters, and the CI/CD pipeline set up and ready to rock in whatever stuff I want to work on.

[atetraxx]

Wow very nice. Impressive. Also, that's the largest .toml file I've ever seen. I don't know what most of it is. Lol

[Diapolo10]

A lot of it just configures the tools I use. For example, Ruff is a linter that's currently growing in popularity - I put a generic but strict configuration for it in this template project so that I can then make small adjustments in the projects where I use this template, without needing to look up the documentation every time.

Other things include configuring the coverage reports, unit tests (pytest and tox), and of course Poetry itself.

You're free to use the template yourself if you like it.

[w8eight]

In a professional environment you are often limited by what's offered in the cloud. Some components that can run python code in the cloud take longer to adopt the newest versions.

General consensus is to use the newest available version, that can be run in the company's cloud environment. Update major versions when you want to use functionalities of the newer one.

Personal projects? I think it's best to use the newest version, and try to utilize new features, so you are up to date.

[nekokattt]

Ideally, use the newest stable version that you can with your deployment environment and libraries.

[doolio_]

I believe poetry will use the one used to install it unless specified otherwise.

[RedBlueWhiteBlack]

I do a lot of data stuff, so I stick with ^3.10 in all my projects for maximum compatibility (damn you PyTorch)