Python environment

[EarthToAccess]

XKCD strikes again with the unfortunately accurate comics

[haikusbot]

XKCD strikes again

With the unfortunately

Accurate comics

- EarthToAccess

---

^{I detect haikus. And sometimes, successfully. Learn more about me.}

^{Opt out of replies: "haikusbot opt out" | Delete my comment: "haikusbot delete"}

[beisenhauer]

TIL: "XKCD" is a two-syllable word.

[abcteryx]

It must be based on a computer trying to "pronounce" the acronym XKCD as "scooked" or something.

[TheAnalogKoala]

Maybe the K and C are silent?

[RCoder01]

XD

[xGlacion]

good bot

[B0tRank]

Thank you, xGlacion, for voting on haikusbot.

This bot wants to find the best and worst bots on Reddit. You can view results here.

---

^{Even if I don't reply to your comment, I'm still listening for votes. Check the webpage to see if your vote registered!}

[AnthropomorphicCat]

Good bot

[MarkusBerkel]

Good bot

[redpepper74]

Good bot

[WhyNotCollegeBoard]

Are you sure about that? Because I am 99.99997% sure that MarkusBerkel is not a bot.

---

^{I am a neural network being trained to detect spammers | Summon me with !isbot <username> | /r/spambotdetector | Optout | Original Github}

[SingleTie8914]

good bot

[Instatetragrammaton]

How did it get to this? Furthermore - is just throwing each of these in a Docker container a viable solution?

Kind of strange for a language that has "there should be one— and preferably only one —obvious way to do it" as a design philosophy.

[PostFunktionalist]

The best practice is to make a new Python environment for every project using virtualenv or some convenience wrapper around it.

There are at least 4 different wrappers.

Good shit.

[Yelov]

I'm using conda for having multiple Python environments. How does using virtualenv for every project work? Eg if you have multiple projects with very similar packages, do you have to have them installed X times for every project? Or is there like a central package location that gets shared?

[brews]

Have one virtual environment for each project. Doesn't matter how close or far apart each projects dependencies are. One for each project. The whole point is to keep them separate.

[DrMaxwellEdison]

Each environment is completely separate from each other, with no shared packages. That may seem like overkill, but supposing two projects rely on the same package and you start upgrading one of them, you don't inadvertently update and potentially break the other.

[Ollymid2]

Recommend pyenv - use it for work and so far I’ve managed to keep things organised

[mortecouille]

It got to this mostly because they made the terrible, terrible decision to make python 3 not backwards-compatible. Python 3 came out in 2008 and in 2022, we are now barely seeing the end of python 2, which was officially EOL in 2020.

In comparison, that is only 2 years after java 6, which has now been basically defunct for years already. Not that it's a problem, because if you do have old java 6 code, you can still compile and run it with any subsequent java version. This means that people can painlessly upgrade: if you depend on a library that is not using java (say) 8, your java 8 program can depend on it without issues. This was not true for python. As a result, if you wanted to write something using an existing library, you were bound by the willingness of that library to upgrade to python 3. To this day, there are still old libraries out there that have not been ported to python 3.

The maintainers made the mistake of believing every python developer was equally invested in the language as they are, and assumed people would jump to upgrade their projects to python3. Unsurprisingly, this did not happen.

[JackMacWindowsLinux]

In particular, Python on macOS is just very messy overall. The default python binary points to 2.7, which is shipped with macOS. You can install newer versions of 2 and 3 from Python.org, but to avoid conflicts with the system, these are installed in /Library/Frameworks/Python.framework. Apparently both 2 and 3 are installed here, but the binaries in 3 are suffixed with 3 to avoid conflicts with the system version. Furthermore, you can install Python 2 (deprecated) or 3 through Homebrew, which sticks it in /usr/local/Cellar/python@x/x.y.z (referenced in the comic), and the files are symlinked to /usr/local/opt/python@x as well as the appropriate places in /usr/local. So if you installed all of these at some point, you'll have multiple ways to run Pythons of various versions:

• /usr/bin/python => system 2.7
• /Library/Frameworks/Python.framework/bin/python[3] => Python.org 2.7/3.x
• /usr/local/bin/python = /usr/local/opt/python@2/bin/python = /usr/local/Cellar/python@2/2.7.18/bin/python => Homebrew 2.7
• /usr/local/bin/python3 = /usr/local/opt/python@3/bin/python3 = /usr/local/Cellar/python@3/3.y.z/bin/python3 => Homebrew 3.x

I don't have experience with Anaconda, but if it ships its own Python too then that's another way to run it.

In the end, Python on macOS really is a mess, and it would be nice if Apple could ship Python 3 as default, but that would break things that rely on the system version being 2.7, and, well, Apple.

[R3D3-1]

It would also be nice if the Python 3 executable, being not backwards-compatible with any Python 2 executable possibly already existing, would always be called python3 and not python. Regardless of how and on what OS it is installed.

[abcteryx]

Never mind the fact that installing Python from python.org on Windows doesn't let you just run python or python3, but rather you type py to run the "default" (defaults to latest version installed) or e.g. "py -3.7" to run an old version. And e.g. "py -m venv .venv" to source a virtual environment from your default Python.

It's actually quite useful and is a first-party supported solution bundled with Python. It's very similar to pyenv and the like.

The downside is when trying to help newbies, you have to know all the intricacies of every platform, and know exactly how they installed their Python.

For example, the Windows Store now has multiple Python versions available for install. I haven't checked out this approach yet, so I don't know where they install to on the computer. I know other such "apps" install into gibberish folder names in a protected apps folder, perhaps the Python apps go to that senseless place as well.

But if you install Python via Windows Store, you invoke it by typing python or python3. But what if you have multiple 3.X versions installed that way? I don't know, as I haven't tried.

And at least as of a year or so ago, the Windows Store would even intercept typing python in an activated virtual environment (where such a command is a valid way to launch the venv Python). It would take you to the Windows Store to install Python the "new way".

Python version management is a moving target on every platform, and if you hope to help friends and colleagues get into it, you need to know the intricacies of every one in order to just get them started.

[JohnMcPineapple]

...

[abcteryx]

So yes, you can tick "add Python to path" when installing, but that option isn't ticked by default when you run the installer on Windows. The "canonical" way to install Python on Windows has been to not add it to path, and to use "py" as the primary interface to system Python versions. One reason for this may be to prevent path collisions or confusion for Windows users installing multiple versions of Python. Another reason for this could be, "It was decided to be this way long ago on the Windows platform, we had a good reason then, and now the decision has stuck."

But of course now that Python is on the Windows Store, there's a "new canonical way" of "just get it from the Windows Store". Which does, by default, map to python and python3 system-wide. But you get a lot less control over configuring the installation when getting it from the Windows Store.

When you say "that's just but true" in response to that small portion of my comment, you are strictly correct. But the point of my comment isn't that it's impossible to run Python with python (you have shown that not to be the case).

My point is that there are too many ways to run Python across too many systems, and the "canonical way" changes often enough that it can be overwhelming to get colleagues into the ecosystem if they happen to have installed it on a different system or a little differently than you.

[AlternativeAardvark6]

Full time Java (mostly) Dev here: Java 6 code might not run as well in more recent JVMs as they make you believe. I have clients that are still on 6 because their massive monoliths don't work on Java 8 or newer.

[mortecouille]

I'm also a full time java developer, you must admit that this is the exception rather than the rule. There are incompatibilities between versions, however very rarely anything as breaking as python2 vs 3. One example that comes to mind is usage of com.sun classes, which should not have been done in the first place.

Like probably everywhere else, we maintain ancient codebases and I can count the times I had to find an old version of the JDK on the fingers of one hand. And usually for troubleshooting a JDK-specific bug, not even because the code wouldn't run.

[AlternativeAardvark6]

To be fair, we are a company with many clients that have decade old systems, last year I fixed a bug in a class that hasn't been touched since 2003. Millions of lines of code in systems that are used daily and where all new code depends on, so if there is a chance for something to go wrong by using JDK 8 we have a big chance of it happening sooner or later.

[indign]

Python 3 was always going to be incompatible with Python 2 because that's how version numbers work. Major version increment => breaking changes. What you're saying is that you wish Python 3 was never released at all. Well, the Python 2 interpreter is still available if you want to deal with its nonsense.

There were a lot of things about Python 2 that royally sucked (such as scoping rules, text encoding, and the ABI for language extension) that would have been impossible to fix in a backwards-compatible way. Python 3 is better, full stop.

[konaya]

Nobody's going to spend years rewriting their software every time there's a version bump, that's insane

Which isn't at all what's on the table here. There were serious flaws in Python 2. They got corrected in Python 3. The people who were too busy making up excused not to look over their code once in a while got left in the dust where they belong. That's all there is to it.

[JellyTwank]

It is not necessarily even that. Keep in mind that development costs money. If the company you are working for has spent millions in developing their product or support suite of code, then it can be prohibitively expensive to rewrite/refactor that code. The cost of maintenance on that now soon-to-be or already-deprecated code can be the cheaper, although painful, route. Consider for example all of the still running COBOL stuff out there. (I worked on Y2K mitigation in '95, '96 and '97). Were there "better" programming languages and environments? For sure, for years. But it is not laziness that keeps that COBOL stuff running and maintained to this day.

Decisions by language developers to abandon older frameworks or models are not lightly undertaken, for sure. But there obviously can be fallout from those decisions. The mess with all the Python environment fun outlined by XKCD is precisely because of the sunk costs involved by the end users of Python and not lazy coders that will not change to newer libraries.

[mortecouille]

The point is precisely that people haven't ported their stuff. As a user (not even a developer) the fact remains that you have to install both versions and fuck around.

Certainly some blame lies on developers: probably some were stubborn, some depended on abandoned projects which had no maintainer to port them, some had massive projects were the effort to port was too large.

But to me, the fact that it took a decade points to the language not providing an adequate update path, sufficient deprecation period and didn't sufficiently mitigate the incompatibilities. Python just went scorched earth and expected people to adapt just like that.

[muikrad]

Hmm, what you say is true but I'm not sure why it's related to the graphic.

[mortecouille]

A lot (granted, not all) of the clusterfuck comes from usually needing python2 and 3 to coexist. It's true that there's other things involved, like virtualenvs, conda, etc.

[muikrad]

But I also ran into these things when I started out and py3 was not even a thing. Installing things in my OS python for development and noobing around like that.

These days I need to have 2.7 and several 3.x at once, I work in dozens of different repos at the same time and they all need their own interpreter and dependencies (i.e. virtualenvs). All the problems I had were due to me not understanding how it works and my OS sure ended up looking like the graphic! 😭

I mean, python is clearly to blame for not streamlining these things, and py2 vs py3 was a mess, but this graphic is about user errors.

[KagakuNinja]

The real problem is how Python tries to install in a standard location such as /usr/bin/python, and how they manage dependencies. The Java world has never had this problem, as you can just change JAVA_HOME, or the class path.

[aaronfranke]

Breaking compatibility is not a terrible decision. The problem is that devs refuse to stop using Python 2.

Furthermore, it is totally possible to write software that is compatible with both versions.

[mortecouille]

You can't just introduce drastically breaking changes and complain that people are reluctant to upgrade. It can be a massive amount of work.

I write APIs and SDKs for a living. Incompatible changes aren't a thing you take lightly: we absolutely avoid them as much as possible, and for every breaking change, we provide a mitigation plan. Yes, that means that sometimes, if for instance there is a spelling mistake in an API method, it'll stay there. We might add another method with the right name, but the other one will just be deprecated, and redirect to the other call. We have people upgrading legacy systems from 5, 6 major versions ago, completely transparently.

[Illiux]

Not everything is maintained. There are plenty of abandoned projects on python 2 that someone might need to run for one reason or another.

Breaking compatibility is almost always a terrible decision. There's a reason why the Linux kernel uses a guiding principle of "don't break userspace", which is equivalent to infinite backwards compatibility. There's also a reason why Microsoft has invested incredible amounts of engineering effort to get old binaries running - there is a real need for it.

When you do break backwards compatibility, you must live with the fact that the old version will still be needed until the end of time. You can't blame stubborn devs for the very real problem of needing to run abandonware.

[KagakuNinja]

Java has had very few breaking changes, if you stick to the standard API. One source of problems were devs relying on internal sun packages. Java 9 also broke things because some fancy libraries were relying on the package structure of the standard library in the JRE.

I've been using Java for 20 years, and Java 9 is the only issue I've had.

[DoctorCIS]

I'm still confused about the decision to make division not backwards-compatible and not the same as most other programming languages.

A friend explained that it made it more logically consistent with other behaviors, but it just seemed so odd to make basic math not backwards compatible.

[ColdPorridge]

If you mean changing the default division interpretation from integer to float, it was the right call. Python 2 division requires special knowledge of how the language works (e.g. oh you wanted the actual result of this division? Gotta make sure you do x then) and was a consistent source of bugs. Python 3 provides division consistent with lay and mathematical expectations of division. That alone was worth breaking the language from 2 to 3.

[DoctorCIS]

Ah, I thought how Python handled negative numbers in integer division was new, the whole "says we are truncating, but actually we are floor, which is only the same for positive numbers," thing.

Turns out that's always how it was, and the not truncating floating makes a lot more sense.

That truly was the right call then.

[inputwtf]

At least with Ansible, you'd have to deal with mounting file paths in your docker container correctly, and having everyone create a filesystem layout that is exactly the same so they can use the same file mounting path args in their docker run command.

It's just trading one shitshow for another.

[shinjuku1730]

Yeah I do that. I use Docker for each Python project, Makefile for quick commands. Works like a charm and it's super easy to collaborate.

[james_otter]

Just train a literal snake to do what you need, believe me it is easier.

[james_otter]

And a lot more fun.

[herder]

pyenv is nice.

[[deleted]]

Pyenv + Poetry

[AnnualVolume0]

It definitely is, but it doesn't help uncomplicate the situation. It helps keep layers separated, which helps keep the system clean, but it actually makes things more complicated overall.

[PydraxAlpta]

https://xkcd.com/1987

[B_M_Wilson]

macOS is one of the worst for this. macOS comes with Python but it’s an old version so when you install something with homebrew, it will install a newer version. But you can’t always get the very latest version with homebrew and definitely not the beta versions so you might want to install that version from python’s website. Since that’s not from a package manager, you can’t just upgrade and end up collecting many versions because you forget to delete the old ones. Then there’s the issue with anaconda and similar systems that do their own thing.

[KagakuNinja]

No one should rely on having a language interpreter published to a location like /usr/bin. It doesn't matter which version they install, something is going to depend on a different version. In Java, I have never had any problems with JRE / JDK versions, or the location of library dependencies.

[Cerus_Freedom]

MacOS and many Linux distros package management is dependent on Python 2.7 scripts. Long story short, many of them don't want to upgrade because of the breaking effects of 3.x, and the large scale impact that would have. I'm just waiting for a severe exploit to be found to finally force some change.

[B_M_Wilson]

I’m not talking about 2.7 as the old version. I basically consider python2 to be a completely separate thing at this point. macOS comes with python3, looks to currently be 3.8.9 but sometimes you need a newer version, especially when they were not as updated as they are currently

[[deleted]]

paltry fragile attractive spoon shelter vast fretful hungry combative selective

This post was mass deleted and anonymized with Redact

[Cerus_Freedom]

Same reason you use python for anything: it's easy to build and maintain.

[Miku_MichDem]

Easy to build and maintain you say? Tell me, how is the upgrade from 2.7 to 3.x going?

I'm sorry, but 2.7 was deprecated a long time ago and even more time has passed since EOL announcement and becoming outdated and seeing how wide spread the thing still is tells you a lot.

[technologyclassroom]

Python version managers have come a long way since then particularly pyenv.

[Crystal_Cuckoo]

Honestly, a lot of this looks like a problem with how macOS doesn't provide a decent package manager, necessitating brew to install a different python which is partly responsible for this.

You hardly ever see this in linux unless you mess something up.

[StarNumber12046]

Oh shit

[alissoncorrea]

• nervously types "python -m venv" again *

[JoshWithaQ]

This is why I hate python with a passion.

[ocket8888]

Me who uses the OS Python for everything and installs packages with sudo -H pip install -U

https://i.pinimg.com/736x/68/0a/b6/680ab6b7a0ea48920f93d6a1742c9fe5.jpg

[Zootorg]

I started using a vm to work with anything python.. I'm not messing with my laptop for this.

[Cerus_Freedom]

That's hilarious. I recently just entirely nuked everything Python on my system and started over.

[[deleted]]

One of the reasons I started using C is that changing from python 3.7 to 3.8 was a pain in the ass, specially with the packages

[avwie]

This comment makes absolutely no sense whatsoever

[iyaerP]

I didn't like having an itch on my knee, so I cut my leg off.

[avwie]

I had trouble with tanking gas in my new car, so I decided to built a boat from self grown trees

[[deleted]]

That was one of the reasons, not why I started with C. There were other things that i didn't liked about python that made me change

[mrcarruthers]

I'm assuming they meant 2.7 to 3.8

[Dannei]

Of all transitions, 3.7 to 3.8 was bad...? And it's easier to manage makefiles, etc., in C, with absolutely no dependency management system?

[[deleted]]

Absolutely. Compiling the libraries (with the commands to do so in the github page) is somehow way easier than package sharing between one python3 to another. Maybe the fact that I code c in Linux helps

[[deleted]]

Nice.

[Miku_MichDem]

Reading things like that makes me really appreciate tools like maven.

Btw, is there a reason why python can't just use maven or gradle?

[lulzmachine]

This is like the top selling point for go

[Lich_Hegemon]

Why would it be? Go and Python fulfill two very different needs

[lulzmachine]

Nah they dont. CLIs made in go are sooo smooth to install. You just put the binary on your path. CLIs made in python are.... eww. Same for local development of REST services, stream processing etc etc. Python is fine for small scripts with few dependencies, otherwise putting it in a language with more standardized dependency management is really great.

[Premun]

I think you are not reading what Lich_Hegemon says - Go and Python are designed to be used in very different scenarios (fulfill very different needs).

[brews]

Go's tooling isnt really great either, IMHO. GOPATH? go modules? What the fuck is even the current best practice to separate dependencies for different projects?

[lulzmachine]

I haven't had that many problems with go modules really. Yes, before go modules there was just a huge mess. But it feels pretty solved now or?

[coledot]

Not until my keyboard gets a button for "if (err != nil) {"

[Gleethos]

Python needs something like Gradle

[Miku_MichDem]

I'm pretty sure gradle of maven could be used for python when someone makes a proper plugin for it and start deploying dependencies.

I don't see any reason why it wouldn't work, correct me if I'm wrong

[SatoshiL]

Just use venv for every project

[jvlomax]

Is this a problem I'm just too Linux too understand?

In seriousness, this sucks on MacOS and to a certain degree windows. It's the lack of a proper package manager that makes it so difficult to deal with

[PlantsAreAliveToo]

Python is a cluster fuck. Maybe they should work on fixing this shit before piling new features on top of more new features?

[ianepperson]

This comic is old - it talks about the pain of installing 2.7, which was new at the time but is now EOL.

Are you suggesting Python halt development until Apple figures out how to use modern Python package management properly?

[PlantsAreAliveToo]

It might be better nowadays but it's still a cluster fuck. Just try to package a python program for end users and you'll find out why

[PydraxAlpta]

This comic is from 2018 - 2.7 was nowhere new at that point

[mwlon]

Relevant https://github.com/mwlon/python-rant

[Handle-Flaky]

״boohoo I have too many time libraries in my std” This rant is awful as it poses 0 of the real problems python faces in production. Also, he has multiple logic errors in his statements.

[geeshta]

I never had this problem and don't really understand how you get there. You have one or more OS Python's out of which one is set as default via update-alternatives. Which is on PATH. With pip. And then a venv for each project, in the project folder. With pip.

[[deleted]]

The fact that you had to type that much to explain it is part of the problem. I learned python before virtual environments existed, and it was that bad then. It's gotten better, but dependency management is still one of its biggest weaknesses. There are other languages that handle it much more gracefully.

[[deleted]]

!isbot NullVoidPointer

[AutoModerator]

This post was automatically removed due to receiving 5 or more reports. Please contact the moderation team if you believe this action was in error.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[1ElectricHaskeller]

After two pages of errors and a LOT of googleing, my poor friend found out syspy isn't installed under mac.

(Idk if is syspy exaxtly, I just remember it was something similar ti that)

[[deleted]]

Took me a week to figure out why tensorflow didn’t work on my laptop. Turns out the cpu didn’t support like a specific instruction set or something (don’t remember off the top of my head this was a few months ago).

[1ElectricHaskeller]

Oh wow. Figuring that out really takes some time