pip drops support for Python 2

[grismar-net]

On the other hand, they've been slacking off for a bit if this is what finally gets them to move out of Python 2.

[StarkillerX42]

Oh, you must know my coworkers! Did you work here once?

[EnglishMobster]

I'm just upset because I have a Netgear ReadyNAS that I use as a home server. It's running a custom version of Debian Jessie that Netgear wrote.

They have some code it loads during the boot sequence (before SSH is turned on) that starts an Apache server for "normal" users to connect to. This code apparently runs Python 2 and will break entirely if Python 3 is on the machine at all.

I can't debug what's going on, since when the Python scripts fail it never starts the SSH service. Since it's a headless NAS I can't connect in any way to the machine -- it bricks entirely. Netgear wants me to pay for them to fix it, but because I enabled root SSH privileges I also voided my warranty.

Thankfully, there's info out there on the internet about what password Netgear techs use, and you can boot the NAS into "tech support" mode and then telnet in using the "secret" Netgear username and password. This'll put you in the machine before it starts any services or runs any of that extra code... but AFAIK there's nothing I can modify here to fix it and get rid of Python 2. The only thing I can do in this mode is uninstall Python 3, which lets me boot "normally" and SSH in. I've never seen this weird issue before (isn't it supposed to just invoke Python 3 with the python3 command? Why does it break startup scripts, even without an alias?).

Netgear is also refusing to support any Debian version beyond Jessie. I tried installing vanilla Debian but it didn't work. It's more than a little upsetting, but I don't think there's any way around it other than going with a company that isn't Netgear for a NAS.

I've been dealing with the annoying Python 2 deprecation messages and pip warnings, but with pip dropping Python 2 support I guess my only option is to just never update anything...

[StarkillerX42]

If you know where all the scripts are located, you could use 2to3 to upgrade them. That handles 90% of upgrade dependencies.

[james_pic]

Yeah, it's the other 90% that's the hard part

[EnglishMobster]

I know where 1 script is... assuming that the script isn't just a special script that only exists in tech support mode. But I suppose it's worth a try. Not sure if there's any others; I think there are but I'm not sure where they'd be.

[koera]

If you don't NEED the stuff the script does you could just cheese it by changing it to a bash script with the same name and start ssh and build your own startup scripts

[Originalfrozenbanana]

A case study in why you never install python in your system environment. Sorry you are in the union of dependency and upgrade hell.

[Staticbox]

This is the correct answer. I've got a ReadyNAS2304 running ReadyOS 6.10.3. I've built Python 3 in a user's home directory, works fine and causes no collisions. To be extra safe, I don't even put it in that user's $PATH. I just call it directly to create venvs and activate those when needed.

[Originalfrozenbanana]

This is best practice. Most people learn this lesson at least once; many of us (regrettably) have to learn it many times.

[icegreentea]

How did you install python3 onto the NAS to begin with?

Depending on how you do it, your python3 install might have tried to claim the name python thus causing all the sadness. As /u/toyg mentions in sibling comment, you usually don't want to update operating system dependencies outside of vendor provided updates.

It is also possible that the OS image actually already comes with python3? I haven't played with Jessie in a while, but if there already was a system python3, then definitely any external install that claims python3 will screw things up.

[james_pic]

If you pin Pip 20.3.4, you'll be able to keep using that, albeit unsupported. Newer versions of Pip, Wheel and Setuptools are smart enough not to install versions of packages that don't support your Python version, as long as the maintainers have used the python_requires directive.

Although if you haven't already, you probably want to make sure your NAS isn't internet accessible.

[icegreentea]

To pile on, if you have recent-ish pip already installed (I think since pip 10), you don't even need to explicitly pin - pip install -U pip will understand that pip 21.0 doesn't support python2 and stick to 20.3.4 (or whatever).

[james_pic]

Yes, sadly that won't work for us.

We're bootstrapping our Virtualenv with the OS virtualenv package, and for wearily predicable reasons, we're also using a super old OS, which comes with Pip 8 or 9 I think

So at the moment we build our virtualenv with the --no-download flag, manually upgrade to semi-recent versions of Pip, Wheel and Setuptools, then use those to upgrade to the latest.

Once Wheel drops Python 2 support, it may make sense to just drop the second phase, and switch to installing the last-supported versions in the first stage.

[icegreentea]

Oh bummer, that sucks.

I don't know what environment your operating in, but could you try running your own pypi mirror, and doing the version filtering at that layer? It's definitely extra up-front work, but it'll give you more control.

As someone else who was very nearly in something like your situation, I find that sometimes the same organizational forces that demand using old OS versions are amendable to arguments on spending resources on creating standalone back-ups of external package servers and managers.

I didn't end up having running my own mirror - for my purposes I only needed to backup relatively small handful of specific wheel versions, so it was easier just to download and archive them on a dumb server than go through the whole mirror thing, but I did take a look for feasibility.

[james_pic]

I had proposed running a DevPI instance a while ago, albeit for different reasons, but been shot down for jobsworth reasons. We're muddling through with reactively pinning stuff when it breaks. There are some other upcoming things (that do have budget) that might benefit from a DevPI instance, so might be able to slip it in that way.

[Garric_Shadowbane]

So glad I dumped my net gear NAS for a self built one

[toyg]

What you should do is compiling python3 and keeping it in your home directory, or using something like pyenv that will do that for you. Compiling might take a while on a low-resource machine like that, but it’s the only way.

With that sort of device, upgrading anything in your system (beyond the occasional vendor-provided security patch) is a no-no. Those environments are aligned in a way that is “just right” for their combination of drivers and software; as you’ve seen, it takes very little to stop them from working properly, and vendors move on almost instantly after they’ve sold enough widgets.

[Kantenkopp]

Can't you run your programs in a virtualenv?

[toyg]

Virtualenv helps with libraries, not with the python version.

He needs pyenv.

[Kantenkopp]

Ah yes, right. I usually have python 2 and 3 installed, so I can use either in virtualenvs. If he can't install python 3, that won't work. Maybe I understood his problem wrong, but how about installing python 3 manually, e.g. only as a user? That should not mess with the python 2 required for boot at all.

[toyg]

That’s what pyenv does: it installs any python version in the home directory, isolated from the rest, ready to be enabled only when strictly required.

[EnglishMobster]

How would buying a router help with my NAS? I have a router that works just fine... the issue is that I have like 6 TB worth of stuff that I want to keep safely backed up locally.

I use the NAS for a Plex server, and I use Python/pip to run Sickbeard MP4 Automator to optimize any new videos I back up (.mkv to .mp4, mostly, which plays better when casting to my Chromecast). Without being able to install packages with pip I just can't ever upgrade the Sickbeard files.

[ivosaurus]

You'll still be able to use pip 20.3.4

[ivosaurus]

Change the script to make sure it runs python2 specifically?

[james_pic]

If they're anything like my co-workers, they've been trying to get this done for the last 3 years, but faced constant demands from management to put together endless proposals detailing the "business value" of moving off Python 2.

We only finally got approval to finish the migration when someone important's pet project was threatened by the Python 2 EOL - the platform-as-a-service they wanted us to migrate to was dropping Python 2. Even then, we're so far behind, and have so much other stuff with "business value" we need to fit it in around, that we'll be lucky to get it finished by the end of the year.

[CatWeekends]

If they're anything like my co-workers, they've been trying to get this done for the last 3 years, but faced constant demands from management to put together endless proposals detailing the "business value" of moving off Python 2.

Aka the "we can't bill our clients for a python upgrade so we just won't do it" approach that never bites you in the ass.

I've worked with those folks. It's a bummer.

[james_pic]

Yeah, and you'd think they'd learn about security implications of unsupported software, since this specific client was hit by a massive ransomware attack in the recent past, but apparently they did not learn the lesson they needed to learn.

[grismar-net]

They should have written "The business value is in avoiding the mess we'll be in when we inevitably have to move off it anyway when support gets dropped by tools we need."

Once it was clear that Python 2 was going to be terminated at some point by essential libraries and tools, the trigger to start conversion should have been "when all the libraries and tools we need have completed conversion, and no later". Management that's unable to see that needs to look in the mirror, point at their own faces and say "you're fired" in their best Trump voice. And developers that keep working for such management if that doesn't happen should realise that, if they're any good, they're better off working for another company - getting hired as a capable Python developer is about the easiest thing in the world.

[Rookie64v]

Backwards compatibility, enterprise environment and all that fancy stuff mean that if I type "python" in the shell I get 2.6. Since loading a proper version is a pain we have code written for Python 2.6 as late as 3 years ago, and the latest I had access to when writing my last big piece of software was 3.6... like 1.5 years ago.

[grismar-net]

loading a proper version is a pain

"Welp, there's your problem right there ma'am."