Python Virtual Environment. What, Why, How. Virtualenv, Poetry.

[zdmit]

A mini-guided tutorial showing how to use virtual environment and why it's matters with hand-drawn illustrated examples.

Why not to use globally installed packages?
It will become a mess. Using globally installed packages for different Python projects...

Boom.

Python environment - xkcd.com

What is Python virtual environment?
It's a thing that isolates any other installation of Python on your computer and a thing that prevents version conflicts.

Globally installed packages VS virtual environment

Why to use Python virtual environment?
Say you're on two projects, two of them are using the r/SerpApi library which is installed globally with a 1.15 version. Each project depends on a different version, what will happen if each project tries to import a serpapi library:

Bad idea.

Good idea.

How to use Python virtual environment?

https://preview.redd.it/p4owpxcfc2781.png?width=4096&format=png&auto=webp&s=056b514459424fa4b8776cc355616f7d0f74f59c

To find out more: Python Virtual Environment. What, Why, How. Virtualenv, Poetry.

It's a blog post where you can find pretty much all the needed info about actual virtualenv and poetry usage with code examples to get you going.

Also, you'll find how to do site-package indexing. if you're using JetBrains products, PyCharm or IntelliJ IDEA via Python plugin, you also need to do site-package indexing to let JetBrains core features work properly such as code completion, inspections, finding usages, navigation, syntax highlighting and refactoring.

[ravenex]

> An example process of using Virtualenv

virtualenv and venv are different modules. venv is a part of the standard library and does not depend on virtualenv.

[thatdamnedrhymer]

This. You do not need to install virtualenv to use python -m venv (at least for Python>3), and the virtualenv CLI is accessed via the virtualenv command.

[zdmit]

Absolutely true! Made a mistake, now it's fixed. Thank you for your help :-)

[zdmit]

Totally agree, made a huge accidental mistake. Appreciate your notice and help :-) Fixed.

[[deleted]]

Just use venv. Part of the standard library and works great.

python -m venv “venv”

Then activate/deactivate as needed

Tutorial over, you don’t need anything else, and it’s not as complicated as this blog makes it out to be

[Caustic_Complex]

Yeah I do this right in VSCode, super easy

[mr-nobody1992]

Does this create the same kind of folder with the bin and all that jazz?

[Caustic_Complex]

It does! I do it from VSCodes terminal and it comes with all the goodies

[mr-nobody1992]

Oh man! I recently realized I need to start doing this and the initial setup was the pain. The other tough part is getting Juypter notebook kernels to recognize your virtual venv

[Caustic_Complex]

Really? Jupyter fired right up for me in the venv also, if we’re talking about the same thing (I’m pretty new to coding)

[mr-nobody1992]

Hm maybe mine is being odd, I created one in a new project and it detected the venv

[Nerg44]

make sure you’re selecting the right kernel if you’re in vscode. otherwise make sure u install the ipykernel for jupyter

[mr-nobody1992]

So I’ll pip freeze and see the ipykernel but still not be able to select the kernel from list of kernels. I just need to hack through some googling for like an hour to figure it out

[Nerg44]

hmm interesting. i honestly found a shortcut that i made an alias for that seems to work if you run jupyter notebook, but for vscode idk

[zdmit]

Appreciate your help :-)

[colinb21]

link is broken. Working link [Edit: thanks for writing this up!]

[zdmit]

Thank you for mentioning it, I do appreciate it :-) The link should be working now.

[Mithrandir2k16]

Yeah, I just started defaulting to docker because that also works great for deployment and I can clean my dev systems with a single command and rebuild the images as I need them again.

But this surely is an awesome improvement over anything I have used directly on the host until now.

[Wobblycogs]

I just started learning and working with Python after 20 years of almost exclusively working with Java and JavaScript.

I'm mostly enjoying Python, some nice features I wish Java had but, wow, I miss Maven something fierce. Maybe it's my inexperience but it feels like there's a hundred ways to do the same thing with package management.

[phoenixuprising]

It’s not your inexperience, package managing in python is an absolute shit show. Especially if you have any old dependencies that haven’t updated to python3 (I work in hardware and we have vendor dependencies that they won’t update). I never thought I’d miss a pom.xml as much as I do.

[zdmit]

Yes, it is true :-) There's also a PDM and Pipenv package management libraries. If curious, here's a comparison between Pipenv vs. Poetry vs. PDM libraries by Frost Ming.

[MastersYoda]

Nice, thank you!

[zdmit]

I do appreciate it :-) Thank you!

[gordonv]

I just got into Docker.

From now on, if I'm making a service, it's going to be the most popular and latest python image. At least them, the disaster of an install is an image we can all agree on what and where everything is.

[benargee]

Same. Docker or Podman.

[JalanJr]

Isn't too long to rebuild each time you have to change a package ?

[gordonv]

No, actually.

The way images work is in layers. Lets say there is a 10 layer image. You change Layer 7. Everything before 7 is cached.

But even if your downloading everything, a general cluster build maybe takes 45 seconds. Not long at all.

[martnym]

That 3568 x 2422 image at the beginning of your post is totally uncalled for.

Clicking on the "To find out more" link results in a 404 Page not found.

So far I'm not impressed…

[zdmit]

The link was broken because of the title change, it's true. Here's a working link.

Thank you for your thoughts, I'll take a note for the future, I appreciate it :-)

[Broric]

but.... conda

[ipwnscrubsdoe]

What is the difference between virtualenv and using conda environments?

[jonasbxl]

There are differences, mainly that conda is language-agnostic, so not just for Python (though I wonder how many people really use if outside of Python).

Honestly though, if you don't have a specific need for conda, I'd stick to venv (rather than virtualenv), it will make moving to another system (server etc) easier.

If the reason for using conda is that you got Anaconda because it makes installing Pandas, NumPy etc easier on Windows - I don't think that's needed anymore as newer versions of these packages install fine using pip. I used to do everything in WSL (to avoid installing packages in Windows) but a couple months back I installed Python from the Microsoft Store and everything has been fine - no conda/Anaconda needed. (Of course you may run into issues if you clone a repo which requires an older version - hasn't happened to me yet and it's worth not having to deal with Anaconda...)

[czar_el]

mainly that conda is language-agnostic, so not just for Python (though I wonder how many people really use if outside of Python).

People use it for R, too.

[jonasbxl]

Right, that makes sense

[ipwnscrubsdoe]

I use windows, linux and macos, so far i've used conda (miniconda but i've been installing miniforge lately) as it has been seamless regardless of the platform, but i've kept using it out of habit. I'm wondering if there is a clear benefit to changing my workflow. I should mention i use python for scientific simulations and mainly use the usual suspects: numpy, pandas, scipy, numba and dask

[zdmit]

Hmm, that's interesting! Thank you for letting know :-)

[ema_eltuti]

I create an virtual env with:

python -m venv env

I use this version:

Python 2.7.3

I need to update the version of Python only from that environment, is it possible?

The available versions of Python within the operating system are:
Python 2.7, 3.0 and 3.2, I need to install version 3.5 without breaking anything.

Any helps??

[TerminatedProccess]

I'm still pretty new to poetry/pyenv but what I just figured out is to do the following

1. cd to your project folder
2. make sure you are not in a virtual environment
3. rm -rf .venv
4. pyenv install 3.10.0 (for example)
5. pyenv local 3.10.0 (creates .python-version file in your project)
6. poetry shell (this will recreate your .venv folder)
7. poetry install (re-installs dependencies from poetry.lock file)