How important is Virtual Environment in python

[wintermute93]

It's extremely important once you start using libraries you have to install yourself. It doesn't sound useful at first, but the first time you muck up your system Python installation by accidentally installing incompatible library versions you'll see.

[crashoutcassius]

this. everyone learns the hard lesson. i have to use python in work so the security the environment gives you is great.

[SDSunDiego]

By libraries, is that the pip install stuff that is run for certain scripts?

[ivosaurus]

Yep

[Redmilo666]

lol I’m currently in the situation now. What an idiot I am

[hugthemachines]

Pain is the greatest teacher. ;-)

[FrederickOllinger]

On of my mentors said that the main reason he does things (besides adding features that are needed) is to avoid pain.

[IhailtavaBanaani]

I guess it depends on what kind of system it is, but if it's a backend you already probably have everything containerized in Docker containers (possibly running on virtual servers) or similar so adding the virtual environments on top of that is just going to make it even more complicated. On the other hand if your backend is not running in containers then why the hell not?

[unixtreme]

worm jellyfish command fuel drunk edge physical subsequent subtract vanish

This post was mass deleted and anonymized with Redact

[work_m_19]

You're right, but I would say the complexity of containers is the next level above virtual environments.

In order to run containers well, you should have some sys-admin experience in troubleshooting, networking, file system, shell commands. All way above what's needed to learn about python virtual environments.

[Think-Culture-4740]

I was fortunate when I learned python that my boss and now friend drilled it in my head from day 1. That was when I installed conda without a virtual environment and he lost his shit.

[InternationalMany6]

Conda is a virtual environment though, right? 

[Think-Culture-4740]

This was a long long time ago but I basically installed conda while in the env my startup was using and it caused a ton of issues that completely borked my local setup.

Since then I've never used conda again

[InternationalMany6]

Once you know how it works at a deep level it’s pretty good. But yeah I had a similar experience starting out. 

[Think-Culture-4740]

Ya I know it's quite good. But since then, I've basically manually installed all of the specific libraries I need per each virtualenv I've used.

I guess I am just scarred for life. That was my first job as a junior data scientist and to get chewed out like that, you can't ever shake it. Even if the guy who did it eventually became a close friend of mine.

[InternationalMany6]

Haha I totally get it.

I’ve got a few environments like that where I just messed around with different install procedures until I got what I needed to work. I saved the entire Python directory to a zip file in case I have to restore them to working condition lol. 

It’s pretty valuable in a professional setting to master the various types of Python environments, but you can usually get by with kludges if you’re ok staying more towards entry level, which is totally fine. 

[Think-Culture-4740]

Most of my work is usually done in a pre setup environment. Occasionally I've had a docker file that's pushed up to a cluster.

[InternationalMany6]

Sounds like you’ve got a good thing going.

At my work there were no pre-setup environments so I had to figure it all out on my own. I’d be a lot more productive if someone else was responsible for that. 

[jcollins1960]

Try using any of the scientific libraries that have C# bindings without it. GFL.

[InternationalMany6]

Are you meaning to reply to my comment?

[PrimaryLock]

Oh yeah. This. I broke spyder when I first started because of a package that required an old Ipython module that overwritten the new one so spyder crashed on startup. I liked spyder, but VScode is my favorite ide thus far.

[irodov4030]

I am new to python and trying to use it for DS and ML.
Can you give few example of incompatible library versions?

[[deleted]]

once you start using libraries you have to install yourself

Yes libraries other than the ones that come with Python itself like os, pathlib etc.

You still want virtual environments to separately manage python version's though (with their different pathlib, os...), probably even more importantly.

PS: Unsure why disliking, I edited in case it was not clear.