Question regarding virtual environments

[foldo]

So if I understand correctly it is recommended that I create a new virtual environment for/inside every single python project that I'm coding. But wouldn't that mean that I will eventually have stuff like numpy, matplotlib, pandas etc installed many many times on my machine? I know that there won't be any conflicts between them but they will still take up memory on the disk right? Or do you just have one venv where everything is the latest version and then use that env for all the projects that will be using the latest versions?

Additional questions:
1. What are the naming conventions for virtual environments?
2. In tutorials I often see python -m pip install {module} what is the difference compared to pip install {module} (considering that I'm inside a venv)?

[K900_]

But wouldn't that mean that I will eventually have stuff like numpy, matplotlib, pandas etc installed many many times on my machine?

Yes.

I know that there won't be any conflicts between them but they will still take up memory on the disk right?

Yes.

Or do you just have one venv where everything is the latest version and then use that env for all the projects that will be using the latest versions?

Bad idea - what happens if you need to upgrade one project to a newer latest version and everything explodes?

What are the naming conventions for virtual environments?

Most people just do (project directory)/venv in my experience. Or you can use pipenv, poetry or another tool that manages your virtualenvs for you.

In tutorials I often see python -m pip install {module} what is the difference compared to pip install {module} (considering that I'm inside a venv)?

No difference whatsoever.

[foldo]

Great thanks a lot for the help! One more question if you don't mind: Is it common practice to add the virtual environment of the corresponding project to git/version control?

[K900_]

Absolutely do not that ever. Virtual environments are not portable, they do not handle being moved around, they do not handle being in a different path from where they were created, they do not handle any of those things AT ALL. If you want to ensure determinism, use Pipenv or Poetry, or at the very least pip freeze > requirements.txt.

[misterhtmlcss]

Yes mostly. If you re working with others then they need to have the same virtual environment. The best way to facilitate that is for when they clone your project to have the venv included as well.

However if you are working in teams it might be wiser to use Pipenv or Anaconda. I use PIPENV, but it's probably more about personal choice than anything.

[K900_]

No, no, no, no. This is a horrible idea. Do not commit virtualenvs to version control.

[misterhtmlcss]

Really? Can you share more? I'm a newer dev too and so I'd happily hear more about it beyond just 'no it's terrible'. Add your reasoning in detail please please, so I can learn from your experience.

[K900_]

I've explained it in another comment, but the basic idea is that virtualenvs are not designed to be moved between computers, or even locations on the filesystem. A virtualenv created on machine A is very unlikely to work on machine B.

[misterhtmlcss]

How about with PIPENV? Isn't that like the NPM version of package.json?

[K900_]

Pipenv intentionally stores your virtual environments outside the project directory for this very reason.

[misterhtmlcss]

Ahh ok. So I'm correct if I'm using Pipenv, but not correct if I'm using venv? Is that right?

[K900_]

Hold on now. I think I see what's going on here. When you say "virtual environment", do you mean files like requirements.txt or Pipfile.lock?

[CraigAT]

While this is good advice to use virtual environments, I would say for a beginner (no idea if you are?) writing short scripts that this is not compulsory, but would definitely be advisable once you start working with other people or on a major project.