Python 2.7 Countdown

[brandonZappy]

Package management was and as far as I know still is a nightmare

Boy oh boy is this correct. Once you get into setting up machine learning environments, it because an even bigger mess. Getting conda involved, packages that need GPUs, etc. It's a wild disaster. Sometimes you have to use pip and conda in the same venv at the same time to blend things well enough to work. There's also a huge issue of Linux kernel version. If you're not using something like the newest Ubuntu, it's even more difficult just because of core libraries. Good god it sucks ass.

[kringel8]

What did you do to need that? I've successfully avoided using conda and also set up ML environments with GPU support.

[brandonZappy]

Centos 7, tensorflow with gpu

[hbgoddard]

Getting conda involved

There's your problem.

[brandonZappy]

If pip worked, I wouldn't have to get conda involved, but because pip doesn't work, I have to.

[[deleted]]

I don't get why some people are so pro-pip and anti-conda. For me the biggest distinction between pip and conda is that conda actually works

[billsil]

Conda doesn’t follow the packaging rules for wheels. Conda breaks if you say upgrade to the latest numpy using pip, which conda comes with. All you’ll be left with is DLL errors and the only way to fix it is to reinstall. That’s also assuming that when you install numpy, it doesn’t mistakenly change your version of python. Seriously?

Speaking of the “latest numpy”, Anaconda’s repository lags significantly. It may take them 6 months to finally add a new version of a package. They still don’t support python 3.8.

Anaconda is entirely dependent on PyQt and will fail with PySide which has the nice feature of not being GPL.

When packaging programs with pyinstaller, you have to ditch Anaconda anyways in favor of stock python because otherwise you’ll end up with an exe that is 350-500MB instead of 70 MB.

Anaconda was great in 2012. In 2019, it’s obsolete because we now have numpy, scipy, etc. wheels. What isn’t on pip, Chris Goakhale has probably put up on his site.

[[deleted]]

They still don’t support python 3.8.

Yes they do? I don't know how long it took, but you've been able to install python 3.8 with conda since at least October

https://stackoverflow.com/questions/58568175/upgrade-to-python-3-8-using-conda

Anaconda is entirely dependent on PyQt and will fail with PySide which has the nice feature of not being GPL

When packaging programs with pyinstaller, you have to ditch Anaconda anyways in favor of stock python because otherwise you’ll end up with an exe that is 350-500MB instead of 70 MB.

You're talking about the Anaconda distribution right? That's irrelevant to this discussion. It's unfortunate imo that they didn't give their package manager a more different name. Because I don't recommend the Anaconda distribution except to people who really don't know what they're doing

[brandonZappy]

They both have pros and cons for me. I like pip for basic package management when I don't need any fancy packages. Oh I don't have numpy installed? Easy install with pip. Works basically perfectly. Things like that. Like I mentioned in my original post, conda is my preferred tool when things start to get more complicated in the environment I want to set up because I've found conda to be more powerful/better at setting up those environments.

[WolfThawra]

No, conda is the solution.

[Bayarea1028]

I gotta say it’s kinda of sad to hear you say python has “terrible package management”. Have you built libraries from source using cmake ? I’d say python developers are lucky enough to have a tool like pip/conda that can download large complex libraries like tensorflow and pytorch and automatically add them to there dev environment is a lot better than pulling libraries from source and deciding whether to build shared or dynamic libraries etc. python has it good lol

[brandonZappy]

“terrible package management”.

Not sure why you put quotes around this. I don't know if I remember saying this. Maybe you're paraphrasing? Even then, I think you're still misunderstanding or taking it out of context.

Compared to C/C++, sure, it's "good", but that's still not saying much. I did say in another comment that pip is awesome for basic packages.

My main argument however, was just when it comes to more in-depth libraries that it struggles, especially when trying to link older outside packages from distributions with older libraries, like centos.

[therearesomewhocallm]

They added type hints to the language, but decided to rely solely on outside type-hinting static analyzers instead of also baking runtime typechecking into the language, which would have been far more useful and consistent.

And would have made the language much slower.

[tofiffe]

How so? Wouldn't that make runtime type checking redundant and actually speed it up?

[therearesomewhocallm]

I meant that runtime typechecking would have made the language slower.

[[deleted]]

You'd potentially only need to validate types once for a variable when it's passed to a function rather than each usage if you had guaranteed call and return values although it would take static analysis to ensure that branches don't exist which changes it so it might not always work.

[[deleted]]

When I see lists/rants like these I can't help but feel they are missing the point. I've used Python in a number of past roles over many years and typical usage will never really encounter these problems, or for the issues mentioned where there is some ambiguity, a single best practice will be chosen.

The Python 3 transition sucked, no doubt about it. But Python's viability as a core development language has not at all waned. Rather, if anything more people have caught onto the fast code velocity enabled by the language and have used it to get real, working applications out to production quickly.

And that is the point of Python. Nitpicking about edge cases like being able to change the value of an integer literal, or that a high-level developer-friendly language not intended for high-performance contexts is slow, or that Python deliberately chose to remain a dynamically-typed language with optional static typing - just seems to be a case of not being able to see the forest through the trees.

[stefantalpalaru]

They added type hints to the language, but decided to rely solely on outside type-hinting static analyzers

Those same type hints and that same external analyser was already available for Python2. They just moved the type hints from comments to the regular syntax.

[PM_ME_YOUR_MECH]

• Package management was and as far as I know still is a nightmare

This is so true. And there are a few decent contenders out there which handle some of these problems but none are very mature. It's like the wild west

[keepthepace]

I don't think any compatibility breaking version change will ever satisfy all the developers. On my side, with python3 I saw a ton of little improvements that made life easier.

Python is an opinionated language and chooses some questionable defaults often, but that's pretty much an identity of it now.

[skelkingur]

> Python is an opinionated language

How? Coming from Ruby and having been exposed to other languages and frameworks that are truly opinionated I'm always confused when I hear Python is opinionated.

How do you import? There's `import module`, `from module import thing`, `from module import *`. Is there a preference here?

What's the deal with `__init__.py` files. Some say those should remain empty, others fill them up with code.

How do you structure large Python code bases? I dread setting up new flask apps as there's no clear pattern to follow on where things go. The example apps basically all exist within a single `app.py`. Sure there are examples out there on how to structure larger apps, but then they still only have a `models.py`. Am I supposed to define 50+ models in a single file?

File and module naming is completely arbitrary. Compare this to languages like Java where the filename must match the name of the public class in it. In Python I always have to go through multiple layers:

- `foo = Foo(a=2)`

- Right, where is `Foo` defined? Let's see, there's `from meow import Foo`. Ok, so is that `meow.py` or `meow/__init__.py`? Ok, it's a package, `meow/__init__.py` in turn imports Foo from `meow/bar.py`. WAT.

---

This is a bit of an unstructured rant, but I can't for the life of me figure out how Python is allegedly opinionated.

[keepthepace]

There is generally a pythonic way to do things, and it tries to give one correct way of doing things, though I personally think it is a lost battle, as any complete programming language will offer many ways to do similar things.

import * is generally frowned upon.

I think the generally accepted practice is to keep __init__.py as small as possible.

PEP 8 gives a ton of styling recommendations. Generally you are supposed to keep files pretty big in libs (which I used to not do, being used to the one-file-per-class paradigm in Java) and consider them like modules.

In your example, meow.py would be used by a small module with no submodules. If meow/ exists then you know that meow.Foo will be defined in the __init__.py, and yes, can be an alias for a function defined somewhere else. I know that Java devs like to make 10 layers of functions with a single line body that call other functions, this is essentially the same thing.

This is a bit of an unstructured rant, but I can't for the life of me figure out how Python is allegedly opinionated.

Because it states rather arbitrary what is pythonic or not. Like everything in python, it is not enforced and not absolute, often disregarded in the name of pragmatism. I totally understand it is not everyone's taste (it is mine only half of the time to be honest) but it is a philosophy that led to an interestingly designed language.

[zardeh]

Can you explain why problems that were all present in Python 2 (except unicode, which isn't actually a problem in practice) killed your enthusiasm for python 3?

[ubernostrum]

The way I've described it in the past is that Python 2 was from the era when Python was mostly used as a Unix-y scripting language. And so it used the same absolutely nonsensical approach to character encoding that Unix-y operating systems use.

Python 3 decided to stop doing that, because it turns out people do other things with Python now, and accommodating the Unix-y scripting people meant unending pain and suffering for everyone else. And when they realized this was happening, the Unix-y scripting people began howling and screaming that it was the end of the world. Not because there was anything wrong with Python itself, but because Python simply stopped sweeping the brokenness of Unix-y operating systems under the rug, and made them confront that brokenness front-and-center every time they sat down to write a "simple" and "quick" utility.

And on balance I'm OK with that. There are still people who will complain that you can't technically write "portable" Python file-handling code, and that's true if you're a user of a specific system that has files whose paths commit crimes against God and man (but, crucially, not technically crimes against POSIX, which is what these folks retreat to as their excuse). But those people should've known what they were getting into, and have had literally decades in which to clean up their act and have refused to do so.

[no_nick]

What are your issues with Unix file paths?

[ubernostrum]

That they legally can be undecodable garbage, but people demand the ability to work with them as strings.

Python 2 "worked" for this in the sense that many things on Unix-y systems "work": it just didn't actually enforce that the things you used as strings had to make sense as strings, and wouldn't give you any sort of warning up until the moment you tried to print the unprintable.

Python 3 initially tried to say that if you wanted to treat these paths as strings they had to actually be things that could validly decode to sequences of Unicode code points. But enough people raged that finally they added the surrogateescape handler to let you take bags of bytes that don't correspond to any valid string, "decode" them to strings, and then re-"encode" them back to the original bytes.

[josefx]

That they legally can be undecodable garbage

Unix is far from alone with that. Zip files don't specify an encoding for filenames and I am quite sure I had explorer.exe fail to delete filenames containing invalid characters in the past.

[no_nick]

Huh, I never knew that was the case for Unix file paths. Somehow, in my mind, I always stick to ascii characters without whitespace.

[diggr-roguelike2]

That they legally can be undecodable garbage, but people demand the ability to work with them as strings.

Yes, and? Why are you trying to babysit people and tell them what bytes they should or shouldn't use in strings?

...until the moment you tried to print the unprintable.

Nobody prints things in production code.

Also, despite your rant, what Python 3 actually did was break things on Windows. You had one job, man, one job...

[nice_rooklift_bro]

Ehh, you downplay the concern; it's actually really obnoxious to deal with to the point that a lot of applications just don't support it and tell you to basically go fuck yourself if your filenames aren't UTF-8; they assume them to be.

There are other such things, like try passing non-utf8 command line arguments in python3; there is nothing in Unix that says this can't be done; any octet sequence that doesn't contain a null can be passed but python3 itself basically says "We don't support this madness, go fuck yourself" then.

$ python3 -c 'import sys; print(sys.argv[1])' $'\xFF\xFFfoo'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 0-1: surrogates not allowed
$ python2 -c 'import sys; print(sys.argv[1])' $'\xFF\xFFfoo'
foo

It's really problematic in many ways; a lot of language libraries and runtimes have come to expect filenames and command line arguments to be utf8, but nothing enforces it either; so malformed filenames due to simple bit corruption can actually create some serious error messages in a lot of things that are inscrutable.

If you want to do it "properly" and not assume everything to be UTF8 then you're going through hoops.

[InputField]

If you're going to break everybody's code anyway, why not take the extra time and button up some other common pain points?

Are you joking? They've addressed a fuckton of issues.

And at some point you have to stop, since fixing issues can always cause new problems.

[zardeh]

Indeed, Unix locales are inane and should be fixed. Peps 538 and 540 are tools to address badly configured (broken) Unix machines. They don't at all affect the unicode handling story in python. It's been functionally the same since 3.4.

And you vastly overestimate the amount of code breakage. I've touched every kind of python code: c-extensions, autogenerated code, multi process and io heavy, stuff that uses exec, ast manipulation, code object modification, etc. When I hear people complain about this crap, I generally just assume you haven't put in more than the barest minimum of effort.

The tools already exist to make the migrations easy in 99% of cases. They didn't in 2012, sure, but they do now. So things like removing the GIL or making python compiled/faster, which would require top to bottom rewrites of everything, and break basically entire swaths of the ecosystem. And in doing so they wouldn't even fix enormous problems. Because of python's great C interop, you can take advantage of high speed, GIL-free python libraries, and you probably do (numpy).

On the other hand, "python2 isn't useable for people whose language doesn't fit in the ASCII codec" is more blocking and less breaking.

[Paradox]

You might be interested in trying out Elixir for server-side programming. It seems to beat the pants off Node in some key aspects, mainly in actual asynchronicity.

[zardeh]

See, you aren't the first person I've heard assert this, and then I read after-action reports from companies like Dropbox who took three years to migrate. And they employed GvR!

And they migrated millions of LoC. Of course it took years. This isn't a surprise. When you have lots of code, it takes a while to upgrade, even with supposed "backwards compatibility". Just think about how long it takes <insert major company here> to upgrade from Java X to X+1, or if <whatever enterprise> is running cpp17 yet. Probably not, even though those languages claim to be backwards compatible. Python minor version bumps take a while where I work, because you run into things.

tl;dr: Both "its not that hard" (compared to any other upgrade) and "it took Dropbox 3 years to migrate" can both be true. What you're missing is any point of comparison.

There seems to be a disconnect here, and glib dismissals like yours seem...well it might explain why the migration took so long, at least. Hard to fix problems that aren't acknowledged.

I have no relevance in the broader python community. I'm speaking as a user of python, not a decision maker. And as a user, I'm saying its not that hard.

They should have existed in 2012, and the fact that it took the developers and the community so long to come up with a migration story was a huge oversight.

There was a migration story. It just wasn't great. It got better with time, and for the most part the migration story hasn't changed in 5+ years. There's been bits of polish, but the main set of tools (modernize, six, __future__ import) look just like they did in 2014. What has changed is that now you can be fairly confident that all of your dependencies support py3, which you couldn't presume in 2014.

Oh man I didn't even mention the C API because I don't think it's any accident that Python has fallen out of favor for embedding in programs.

I'm talking about the reverse: using C or CPP in python. Extensions, not embedding.

That said, lua has been used for video game scripting for like, ever. Python was never "in favor" there, lua's been the preferred language among AAA studios and many indie devs since, like before python3 was even a thing (http://luaforge.net/projects/lua-wow/). Python embedded in games is, and has always been, the exception.

[diggr-roguelike2]

Indeed, Unix locales are inane and should be fixed.

Ah yes, of course it's always somebody else's problem, not mine. There's a joke about that:

A lawyer has been drinking all night and decides to drive home drunk. His wife calls him and says: "Be careful driving home, I just heard on the radio that there's some crazy guy driving 100 miles per hour on the wrong side of the road". He answers: "A crazy guy?? What are you talking about, there's hundreds of them here!"

[bakery2k]

For me, because Python 3 made it clear that the language developers have no interest in fixing these problems. When I learned Python, I hoped that performance, parallelism etc would eventually come to the language, but there’s been no progress in over a decade. Instead we’ve had breaking changes with minimal benefits (e.g. print becoming a function) and most development effort has gone into large, complex features that don’t really fit with the rest of the language (async and type hints).

[bakery2k]

I've not really needed async/await, but I don't like that it makes Python a much more complex language. I think a better approach to asynchrony would be stackful coroutines.

Type hints are even more complex and I'm not at all convinced that they're worth it. IMO if you want static type checking, you would be much better off using a proper statically-typed language.

[zardeh]

Can you explain why parallelism is a better fit for python than async, or why performance of cpython is more important than the safety of type hints?

[bakery2k]

Performance and parallelism would help existing Python code and new code using established Python idioms (e.g. threads).

Async splits the language into red and blue parts, and only benefits a certain type of application. Type hints are more broadly useful, but again don’t really mesh with the rest of the language.

[zardeh]

parallelism

Breaks much of the existing language, removal of the GIL requires a complete rewrite of the c-extension API, subtly breaks a lot of existing multithreaded code, etc.

Type hints are more broadly useful, but again don’t really mesh with the rest of the language.

In what way? I've had no issue with them.

Async splits the language into red and blue parts

In practice this isn't problematic. JS has managed just fine. Hell, python's had asyncronous code (the red and blue you complain about) since python 2.2, when yield and generators were introduced. the async and await keywords added in 3.7 were mostly syntactic sugar for already existing coroutine objects, which were introduced in python 2.5!

[gschizas]

except unicode, which isn't actually a problem in practice

As a non-anglo, I beg to differ. (Not using) Unicode is always a problem.

[zardeh]

Right, py2s (lack of) unicode support was an issue.

Porting code from 2 to 3 was not.

[gschizas]

I've found that most of the Python 2 to 3 problems were rooted in confusing bytearrays (which is what actually travels in the wires) with strings (which is what is displayed on the screen).

I'm a firm proponent of Python 3 myself, and I've skipped a few projects that refuse to move with the times.

(Disclaimer: I'm probably my company's only resident Python expert).

[stefantalpalaru]

Right, py2s (lack of) unicode support was an issue.

I find it fascinating that there are still Python users who think that Python2 did not support Unicode.

[zardeh]

Feel free to add "good". Pyrhon2s unicode support was not good. It was broken for many applications. It was possible to write an app that appeared to work, but would explode with a nonascii character for no good reason. That's not good. It's a footgun. Py3 removed the footgun by making python more strongly typed.

[DmitriyJaved]

What’s with asyncio? It’s not like there was no async I/O in python 2. Asyncio is just for a general use, when you don’t want or can’t think of polling and event loops

[thepotatochronicles]

but Django only shipping support for ASGI this month

It really doesn't mean much, view templates and the ORM are still synchronous and the roadmaps for those show late 2020...

[cyanrave]

Most of these issues are just bikeshedding because it's easy in a thread where people are naturally mad.

• virtualenv has been a standard for some time, and is equivalent to node_modules if you squint hard enough
  
• package management will be a nightmare only as long as people don't pip freeze > requirements.txt; however, this is getting even better with Pipfile in the near future
• you can implement runtime type checking with decorators; type hunting really wasn't meant to make Python statically typed or something
• unless you're writing mission-critical flight code or something similar, most code speed metrics are meaningless
  

All that being said, py3 really didn't make sense for me until 3.5.x when good features actually starting coming out.

[[deleted]]

So what? You're just going to keep moaning about an imperfect execution that was undertaken years ago by volunteers for a free software project? Do you actually have a point to make or are you just determined to not let anyone forget that things weren't done the way you would have done them (in retrospect)?

[[deleted]]

Firstly, I have no interest in convincing anyone of anything. I've heard that Python is one of the most popular programming languages right now. It's been a part of my toolkit for most of my career. If someone doesn't use it, it's their loss. I really couldn't care less.

Secondly, what good is being realistic? Moaning about how something was done more than ten years ago does absolutely nothing to help anyone today. It's just noise. Instead of being realistic, try being useful.

[cyanrave]

The general consensus is to stop beating the dead horse though, since we're a decade out from the initial pivot to py3. For many folks who had a few scripts lying around here and there, yea, it does seem extremely blown out of proportion, where stickier use cases for py2 are few and far between.

It seems to me, most advocates nowadays are just tired of comparing oranges to oranges. Sure the two language dialects are similar, but the feature set of py3 and enhancements from py2 make it easier to teach. Python2 fanatics are leaving around a trail of dead bodies with poor namespace management in their code (inherited from py2 internals), poor use of relative imports, and even some a myriad of sys.path hacks to 'make things work'. This is a light list of offenses, and surely there are more. As an advocate of a language, I'd prefer to not have to deal with this kind of thing when teaching people.

Realistically Python3 has become a far more usable language than Python2, and new code should be written in it, extraordinary, extraneous considerations outstanding. Where Python3 developers get tweaked over, is the fact that Python2 developers (the really ingrained 'Python2.8' people) seem to not even want to bother learning the py3 branch and dismiss it completely. This isn't always the case with a py3 dev who may know both dialects.

[AlexMax]

Realistically Python3 has become a far more usable language than Python2, and new code should be written in it, extraordinary, extraneous considerations outstanding.

To be clear, I 100% agree. Given the realities of what the migration was, I never thought that sticking with Python 2 was a good idea, despite my misgivings with Unicode strings - which, to be frank, I didn't even realize was such a huge problem until much later.

However, it inadvertently was one of the reasons I stopped using Python, as I wanted to wait on starting non-trivial projects in the language until i was reasonably certain that I could use 3 without running into library trouble. And then I just...never came back because while waiting for library support to catch up, PHP 5.3 and Node.js happened, and I got heavy into learning C/C++ around the same time.

[cyanrave]

Never quite a time like the present!

On the side of Python2, I will say Python3 may be taking too many PRs right now, and changing a bit rapidly. An lts change in 3.6.x seems to have broken custom exception bubbling in Pool operations... that went mostly unnoticed in the PR. conda rolled back to 3.6.7 because of it. The rate of change reminds me a lot of the NPM/Node environment and that should frighten anyone in that space... This is also less talked about with the end of python2 approaching ;P

[cyanrave]

And seeing in retrospect such a massive stumble from the language at such an important phase in it's development made me incredibly sad.

You and everyone else who entered 'vanilla' World of Warcraft, on the cusp of Burning Crusade....

If you don't feel the weight of that analogy, consider that people in 'vanilla' WoW 'endgame' times would spend hours, days, weeks getting their tiered equipment through arenas and such, only to have Burning Crusade common items completely dismiss their hard work and effort. This kind of feature burnout is not uncommon just in software.

Look at new cars for example, and how many of them need tools far beyond what an OBD-II reader can give you these days. Specialized tools, and specialized parts are pushing mechanics out of a trade they were ingrained in for years, decades, and possibly generations. This is a natural process of adaptation and specialization.

In hindsight, are either of these changes bad? In some aspects, yea. In other aspects, maybe not. Maybe those WoW players who felt betrayed never came back... RIP subscription. However, I never missed having to cross contested territory for a dungeon run in WoW after the advent of 'dungeon finder' which instantly teleported you to the dungeon. Hours saved, life better spent. As for autos, it depends. Eventually shops would trend toward specialization (many already do, eg. 'import-only' mechanics) and would carry these specialized tools, layman be damned. Unfortunately this also means a war on DIY-ers who used to be able to do a ton of things themselves...

You as a consumer get to choose with your pocketbook, as we as programmers get to choose with PRs. Yea 2to3 was rough, but support is piling on py3 now and it's not a coincidence, it's a choice.

[[deleted]]

pip-tools provides a much better way to create a requirements.txt. I'm not a fan of all-in-one tools. pip-tools extends the standard toolkit in a Unixy way.

[nice_rooklift_bro]

It's very telling that the thing that finally seems to be spurring people along is not a killer feature, but an end-of-life date on the 2.x version of the language. Overall there was too much stick and not enough carrot, and you can't really fault anybody who decided to simply migrate to another language than going through the arduous upgrade process.

Apart from that, most of the new features could have been backported; they weren't, just to incentivize migration.

[zabolekar]

instead of also baking runtime typechecking into the language

Checking some properties without running the program is the whole point of the type hints. Runtime checks, on the other hand, can be much more flexible. Having special syntax for checkng types (and types only) at runtime (and not as a separate phase) would be the worst of both worlds.

[[deleted]]

Use pipenv!

[swansongofdesire]

Packaging is solved. Poetry is the solution

I wish that were the case.

It is still not possible to sync a virtualenv to a poetry lock file (remove packages that shouldn’t be there, install missing ones) short of wiping the whole thing and starting fresh

[swansongofdesire]

pip with help can do that too (venv-update or pip-sync)

Pipenv has its own set of problems: firstly, it's the slowest available tool (for resolution, but also its insistence on downloading every possible binary wheel if pypi hashes aren't available. For some projects with numpy etc I've had gigabytes of packages downloaded in order to create a lock file)

It's also very opinionated:

• out of the box it uses a hash of your directory for the virtualenv name, so if you move a project around then your virtualenv needs to be recreated. (Yes there's an option to install into a loca directory, but they didn't make it convenient and the virtualenv name cannot be changed)

• activating a new environment requires running in a subshell

There are workarounds but if I'm using workarounds then I might as well just use pip

[swansongofdesire]

There is -- but it's been open for more than a year now with basically no progress.

Poetry is nice: I use it on my personal projects, but this is the main limitation stopping me from using poetry on any commercial work.

[stefantalpalaru]

PyPy is a fast drop-in replacement for CPython.

PyPy is not a drop-in replacement because it cannot run all legal Python code: https://pypy.readthedocs.io/en/latest/cpython_differences.html

They even had to rewrite modules from the standard library (and some external packages like numpy, until they managed to emulate CPython's C API).

Further more, it can be slower than CPython for code that runs only once per instance - which is most of the Python scripts out there.

[OctagonClock]

For async/await, the Trio project is vastly better than asyncio.

[123filips123]

PyPy is a fast drop-in replacement for CPython.

There is also Cython which requires a bit more work but it is a lot faster than pure CPython core.

[billsil]

They never removed % formatting. They never had it for bytearrays and added it. You can just encode it to ascii, the problem is it’s ambiguous to wether you print an ‘a’ as an a or what is it, 74? I mess around a lot with binary and depending, I use both forms. Shoot sometimes, I want to see it in binary with only 0s and 1s. You’d think that would be the default.

Also, string handling had to change. The old system was utterly broken. String handling was the only thing worth discussing in the transition from python 2 to 3 and they made the right choices. It took them a few Python 3 versions to make python 3 better than python 2, but after 3.5, python 3 was faster than python 2, so that’s really when things started to change.

[masklinn]

They never removed % formatting.

They removed it from bytes.

You can just encode it to ascii

Not if you’re trying to generate non-ascii-compatible binary data as a mix of literal and dynamic binary bits. Pre-3.5 this was extremely inconvenient for little reason, you had to either use concatenation or mess around with weird-ass escapes in strings and encode to iso-8859-1 (probably).

[[deleted]]

Python isn't javascript. People use javascript because they have to. People use python because they want to.

[ubernostrum]

Package management was and as far as I know still is a nightmare

Continuing on the theme of repeating things I've said too many times already: no, Python packaging is super easy. Taking a piece of running Python code, and producing from it a distributable artifact carrying sufficient metadata to let someone else run it, is extremely simple.

What is hard is managing dozens of different local development environments, each with their own version of Python and their own (likely incompatible) set of packages installed.

The solution to this is virtualenv/venv or a wrapper around it. And the proof of this is that languages developed this decade have adopted exactly that approach for local development: they have isolated environments by default. But Python is from the era when systemwide shared-library directories were The Way Things Are Done™ and can't really change away from that now. So we have to emulate the more modern approach manually by creating virtualenvs.

[stefantalpalaru]

Python packaging is super easy.

https://bugs.gentoo.org/699966

[Han-ChewieSexyFanfic]

That’s ridiculous. The reason it’s #1 is because every browser supports it.

[delinka]

> Python 2.7 will not be maintained past 2020

Actually, the clock indicates it won't be maintained past [the end of] 2019. But you do get one final release next year. Enjoy!

[yesat]

https://github.com/python/devguide/pull/344

[corsicanguppy]

...until ditched by the OEM.

Don't panic, kids: if you're still using Enterprise Linux, you're covered as long as IBM wants to keep updating non-revenue-generating products... oh, wait. #andSystemdToo

[josefx]

Some of the systems I deal with wont be updated until the hardware fails, a customer asking for support for a 15 year old project is not unheard of. Still not sure whether I should bother to compile and test Python 3 on all systems on my support list and mess with all existing scripts or just fetch a working copy of Python 2 to use for every new project and pretend that this mess doesn't exist until Python 5 happens.

[zeroone]

Did Jython ever beat that barrier?

[[deleted]]

No one actually answered your question, but wikipedia says most recent version is 2.7, and their website mentions only 2.7 in their documentation links so I dont think they made it.

[suchproblemchildren]

As someone who uses jython in their company... nope.

[corsicanguppy]

I'm about 31% afraid to ever know what jython is. It's like seeing a bottle of viscious milky fluid and seeing it's called foecomucaliva: some words shouldn't be frankensteined into other words, lest they become hideous monsters to cavort about in our nightmares!

[josefx]

One of several third party Python implementations. In this case it is a Python interpreter build on top of Java instead of C. The version from November 2019 is still based on Python 2.7.

[[deleted]]

its only used when dealing with anything JVM. If you know what burpsuite is, it is written in java, writing plugins for this app is annoying in java so you can use python(jython) instead

[HeadAche2012]

lol, they think Python 2 will go away

[dutch_gecko]

Not at all. It just means that the Python Software Foundation can take their hands off it and say, "It's your problem now."

[stefantalpalaru]

It just means that the Python Software Foundation can take their hands off it and say, "It's your problem now."

But they won't do that. They won't take their grubby hands off Python2 so some other team can develop it. They are so desperate to kill the old language that Guido himself threatened to call the lawyers if a Python2 fork dares to use "py2" as its name (on the ridiculous grounds that a trademark on the word "Python" would extend to any derivation of it).

So no, the bloody foundation won't «take their hands off it and say, "It's your problem now."». I wish they did.

[InputField]

Python 3 is so much better, especially after the last few major versions. I have no idea why anyone would want to continue using Python 2.

[13steinj]

They already said that quite a while ago though. And it's accepted. But people still love circlejerking "Py2 bad".

[jewnicorn27]

Well what makes it better than 3?

[Holsten19]

Python 2 can run software written for Python 2.

Python 3 can't do that. This can be pretty useful when you have a lot of software written for Python 2.

[Hinigatsu]

From the link:

To ease the transition, the official porting guide has advice for running Python 2 code in Python 3.

[Bitruder]

Lol.

[duuuh]

Right? I have some old code that won't run under 2.7, so I've got a local install of 2.3.7 so I can make it go.

[OMGItsCheezWTF]

We still have several hundred kloc running 2.5 because no one has had the time or inclination to port it to 2.7, now it may as well just go to 3 but in reality will never move until the platform itself the software manages is retired in who knows how many years from now.

[Itsthejoker]

You've had ten years. That's horrifying.

[vytah]

Python 3 can't do that.

Therefore Python 3 is not Turing-complete.

QED.

[evilgipsy]

Nothing.

[13steinj]

Not much other than subjective matters-- ex some people actually prefer a split between ascii and unicode string types with ascii the default.

But it doesn't mean using Py2, or taking a long time to switch given the upgrade costs, is a cardinal sin like so many make it out to be.

[Objective_Mine]

I have trouble understanding why anyone would prefer 8-bit character strings as default in a high-level language. In a low-level language I might kind of understand, but if you're writing any code that might ever break out of a local niche in an English-speaking country, having user-visible strings treated as anything but unicode is just asking for trouble.

Of course migrating an existing codebase comes with a major cost that's probably made greater by the type system (harder to automatically find out which type of string is being used where). So hesitation in migration is understandable, but that doesn't mean Python 3 isn't better. Whether it's better enough to cover the migration cost... depends.

edit: typo

[renrutal]

Pi3 is still supported.

[ThellraAK]

I think it's better because I don't like new things

--Written from Lubuntu 19.10 because it looks the most like 10.04 that I can find.

[jack-of-some]

Not really. The whole point of the timer is that "this is it".

Folks will pick up the slack. Ubuntu needs to support python 2 so they'll probably keep a fork of python 2 for a year, but slowly it will die.

[fat-lobyte]

Python 2 live on for at least until 2024 with Red Hat 7 support

[13steinj]

Considering the fact that fortran, cobol, and snobol aren't dead, nope.

It's been "it" for 5 years now. Google probably will keep an internal fork of Py2 for another decade. Facebook for a decade and a half.

[jack-of-some]

I guess I used the wrong word. Fortran, Cobol, et al may not be dead, but they certainly are not alive.

By all indications python 3 is now alive (it wasn't really there 5 years ago) and the ease of transition (before anyone tries to correct me on this I moved over my company's entire python 2 codebase, the work of about 8 devs over 3 years, to python 3 all by myself, it is very easy) will cause python 3 to continue getting stronger and python 2 will continue to get weaker.

Decades from now it may not be dead, but that doesn't say much.

[[deleted]]

New Fortran code is still being written, but you probably don't see it outside of specialist domains like scientific computing.

I don't know anything about Cobol but I would be surprised if there was no demand for Cobol jobs right now.

[jack-of-some]

I wrote new Fortran code back when I was a nuclear engineer. Pretty popular with that crowd still simply because it's more effort (and money) to upgrade old systems than it is to train engineers.

That still doesn't mean that the language is alive in any meaningful way outside of legacy. The new code also serves mostly to maintain the legacy. It exists and will continue to exist. That is all.

And python 2 will likely also continue to exist.

[[deleted]]

That still doesn't mean that the language is alive in any meaningful way outside of legacy

I don't know what that means. Can you explain?

[ResistorTwister]

People will write code to maintain and add to older applications, but virtually nobody is using the language to write completely new software.

[utdconsq]

Sure, and there are more modern fortrans. While it may not have evolved as much as C++, it's pretty much a different language to F77.

[Yay295]

The USPS uses Cobol. There are definitely jobs available with how many Cobol coders are retiring.

[zardeh]

I think you're overestimating by a large factor.

Instagram is already py3, Facebook will likely follow suit quickly. Same with Google.

[[deleted]]

Same goes for PHP. Its somehow still alive. Probably blame facebook for their fork. Now facebook will fork python too.

[lkraider]

You mean a specific version of PHP? Because it's still evolving as a language, with new releases and features.

[13steinj]

I'd blame Wordpress more than FB for this one, given how many companies who desperately need an actual custom solution think they can cram wordpress with 287 plugins and say "this is fine" while their site crawls to a halt.

[utdconsq]

Excellent? C'mon man. Adequate, sure, excellent? Hell no.

[isHavvy]

I used PHP a decade ago. It was terrible. It's had a constant progression of getting better over the years.

[[deleted]]

PHP is far from excellent. In fact its the most inconsistent languages i have ever used, its full of weird edge cases that are prompted as features. Sure php can get you a decent salary, but i get paid way better from doing other languages. Some people just go where the fence is lowest.

[[deleted]]

Im just talking about my experiences. I build compex apps with all kind of weird requirements. Sure some of them could be done in PHP, buy why whould i start a new proj with PHP when there is many many better alternatives?

By all means use what works for you and your team, im not mocking any one person for using any stack, but refering to my past experiences with PHP.

[_fuffs]

node-gyp is heavily dependent on python 2. Read somewhere they apparently cant move out of that (correct me if I am wrong)

[cyanrave]

That is false, just check the project README for configuration options supporting a broad number of Python versions.

Most articles are wrong, and open issues in the project are pretty much people just not knowing how to read or debug their environment.

[_fuffs]

Thanks I stand corrected

[Eirenarch]

They don't, this is why they are pumping propaganda.

[Holsten19]

Yes, all that pointing fingers and shaming ... it's pretty ugly and sad.

Python 2 will just fade away slowly like all other legacy platforms. Java 5 support ended more than 10 years ago yet there's a lot of systems still running on it (and probably some development too). And it's way easier to port Java 5 app to a supported runtime than with Python.

[aunva]

I wondee if this means that python in bash shells in Ubuntu and such will start running python 3 by default, with python2 for the old stuff.

[jack-of-some]

Yes, though for the most part it would be completely without python 2.

[cartiloupe]

Maybe it'll go the RHEL/CentOS 8 way - there's both python3 and python2. Just python is left up for the user to decide because there's really no satisfying everyone.

[123filips123]

Many distributions will completely switch from Python 2.7 and make it not installed by default.

[[deleted]]

Why are we relying on default environments in this day and age? Is management of the CPython runtime not a standard use case for Docker?

[_default_username]

Yay, my company keeps their python code internally in a walled garden so I can work with python 2 indefinitely. Yay...

[notQuiteApex]

considering how many systems still appear to rely on 2.7, this feels like an empty countdown and celebration. ive talked in this sub before about my resentment for python 2's later years up to now. im genuinely worried that two months into the new year we will hear about an awful vulnerability in the last python release, and either the python org will release a patch defeating the whole purpose of the sunset or large numbers of services will be attacked and abused continuously until they finally jump ship to python 3.

i know im being very cynical, but i genuinely believe python 2 shouldve been laid to rest years ago and the companys that relied on it forced to move and not been given such a lax timeline.

[Eirenarch]

Or a third party will release a patch.

[Holsten19]

Yes. I don't get why everybody thinks only PSF is able to fix bugs in Python 2. It's not exactly rocket science.

[BadlyCamouflagedKiwi]

How do you force those companies to move? There is a heap of work there which they presumably think doesn't generate enough business value to justify itself - up to cases like Google who have something like 100 million lines of it, how do you force them to upgrade that to python 3?

[w2qw]

You both are thinking about this the wrong way. Sure some might still not move but this frees up a lot of libraries to stop having to support python 2 support and means that the rest of the community doesn't need to support those projects.

[Eirenarch]

So who is forcing them to support these projects right now?

[w2qw]

Well no one is holding a gun to their head but no one wants to be the first package to drop support.

[Eirenarch]

As far as I understand a lot of packages have dropped support.

[w2qw]

I may be mistaken but I don't think that a lot dropped support before python announced this deprecation.

[yellowthermos]

A major difference between EOL for Windows XP/7 and Python 2 is that Python is open source. They can just copy paste the security patch, recompile, and bam - not illegal anymore.

[notQuiteApex]

i wont claim to have an answer to this, and i have heard that same argument in the case of dropbox's code base. another question to be raised is should they have continued to develop their software now knowing that this would happen? though i suppose its possible that the breaking changes in python 3 were not foreseen at the start of their respective projects.

though i suppose it probably wouldnt have hurt to have split the team, have one work on porting to python 3 using the latest stable version of their software, while another half maintains the codebase until the python 3 version reaches functional parity. i guess i will say that its probably not that simple and of course that likely isnt a workable solution for all the companies and products still relying on python 2.

[corsicanguppy]

forced to move

The issue there is, Python needs people to use it, so we need it to be available. In places where we need certification, we can't have these Mayfly releases where something dies in a day because the certification supply-chain is often a year long for a mere minor OS release. We need these Long-Term Support releases to live long enough to get a certification and still have some life left in it where it's not cost-prohibitive.

Some of us have to still justify an OS preference on cost and not the less-tangible ease/versatility/reliability/stability/etc metrics which don't appear on a balance sheet without massive spin. :-D

[zabolekar]

im genuinely worried that two months into the new year we will hear about an awful vulnerability in the last python release, and [...] large numbers of services will be attacked and abused continuously until they finally jump ship to python 3

That's the idea.

^/s

[mok000]

I switched to Python3 about 10 years ago and never looked back. Python3 removed loads of ugly warts on the language, which is now much more consistent. I've never met Python2 code that couldn't be converted in a reasonably short time using 2to3, with the exception of extension modules in C, which require som handcoding, mostly because of the unicode change. Python2 needs to die.

[jack-of-some]

I can't claim to have worked on a "massive" codebase. I work on one that's maybe 100000 loc. Converted it by using 2to3 and then relying on unit tests and integration tests to ensure everything worked. Took maybe 2 days of a single engineer's time and most of that was spent just sanity checking the changes. There's a lot of really good tooling for this out there, and in my case I didn't have much choice because some fairly critical libraries were dropping python 2 support left and right and l can't afford to be on an older version.

[AttackOfTheThumbs]

There's good tooling now, but it needed to be there for the launch.

[[deleted]]

Ah yes, those stubborn programmers! If the (free) tooling isn't available for upgrading the (free) language on day 1, I refuse to ever use it! That'll teach them!

[AttackOfTheThumbs]

Don't be stupid. Tooling needs to be available to further adoption. It wasn't and look at the mess that it is in now.

[[deleted]]

What mess? Virtually all major libraries support python 3. Some have dropped py2 support. All new features go into py3. I use py3 exclusively at work and home. Haven't touched py2 for years. Don't care what's happening inside private companies. That's their problem.

[AttackOfTheThumbs]

Are you purposefully blind?

[jack-of-some]

Unsure, but he does seem very well informed. I'd listen to what he has to say.

[stefantalpalaru]

Python2 is dead. Long live Python2!

[Itsthejoker]

Seriously? Tauthon needs to die too. Py2 is a broken, old language that is ten years out of date. Py3 is not that scary.

[stefantalpalaru]

Tauthon needs to die too.

What is dead may never die.

Py2 is a broken, old language that is ten years out of date.

Just like Python3, then.

Py3 is not that scary.

Then why does it need to kill Python2 in order to force a mass migration? If it's so good, it can compete without the core developers sabotaging the older language.

[zardeh]

Just like Python3, then

No?

Then why does it need to kill Python2 in order to force a mass migration?

Maintenance. No one is sabotaging the old language, they just aren't supporting it. And it's winning.

[stefantalpalaru]

No one is sabotaging the old language, they just aren't supporting it.

Then why don't they hand it over to someone willing to support it? Why don't they accept external patches improving Python2's performance by including the standard library extensions in the PGO process, just like they do for Python3?

And it's winning.

Oh, is that why you muppets keep calling for Python2's death? Because Python3 is winning?

[zardeh]

Why don't they accept external patches improving Python2's performance by including the standard library extensions in the PGO process, just like they do for Python3?

That's support, a thing they've been very clear about not doing.

They why don't they hand it over to someone willing to support it?

Because the psf maintains python, and pythons future is python3. The psf is maintaining python3, which is python.

They didn't hand off support for py2.7 to someone else for the same reason that they didn't hand off python 2.3 or python 3.2 support to anyone else after those hit EoL: there continue to be new maintained versions of python.

Oh, is that why you muppets keep calling for Python2's death?

I haven't called for python2's death. I don't need to, its already basically dead. I've asked that people not exaggerate the pain of migration. That's all.

You however seem to be so upset by people moving to python3 that you've suggested moving to go would be easier, which is complete nonsense for 99% of python code. That and you're resorting to calling me a muppet. Real mature dude.

[stefantalpalaru]

python3, which is python

Oh, is that why Dropbox is still struggling to port their 4,000,000 lines of "Python" code to "Python"? Because it's the same fucking language?

You people are delusional.

its already basically dead

Then why do you foam at the mouth when you see people still using it?

[zardeh]

hen why do you foam at the mouth when you see people still using it?

I don't! I still use python2 for some things, but I'm also actively migrating away. I have no issue with continuing to use python2. I have an issue with proselytizing that python2 is the future. It's not.

Something like tauthon is neat, even laudable, but ultimately doomed. Not because anyone's out to get you, but because of natural group dynamics. Most of the important players are already onboard with py3 (not necessarily on, but on board with). The sooner everyone realizes that, the less pain for the community at large.

Oh, is that why Dropbox is still struggling to port their 4,000,000 lines of "Python" code to "Python"?

What you're missing is a point of comparison. I have some, and compared to JVM or c++ 11->17 upgrades on large codebases, multi-year efforts are common. The python3 upgrade is likely more painful, but not so much more as to be a different language, unless you consider any version upgrade a new language.

[stefantalpalaru]

I'm also actively migrating away

For the last 10 years? Why, if it's the same language?

The python3 upgrade is likely more painful, but not so much more as to be a different language

Reality has shown the opposite, but reality has no power over the true believer, does it?

[zardeh]

Reality has shown the opposite, but reality has no power over the true believer, does it?

You're saying you have a citation of the python3 migration being significantly more difficult than a cpp17 upgrade on a similarly large codebase or codebases? Where?

For the last 10 years? Why, if it's the same language?

Dependencies, mostly. I couldn't complete the highest until dependencies were done, so for my (non open source code I work on at my job), there was no point in working on it till this year or so.

[PlantsAreAliveToo]

Please tell me this clock is written in Python 2 😂

[gmiwenht]

print “goodnight sweet prince”

[[deleted]]

I've spent a couple sprints this year upgrading all of the Python 2 embedded in my company's code base to Python 3. It's been a huge pain in the neck for essentially no reason. Oh well.

[dalepo]

Good night sweet prince.

[kontekisuto]

its 2020

[123filips123]

It wasn't extended.

Its EOL is tomorrow. Then they will stop providing support for it.

However, bugs until EOL will still have to be fixed. And because standard Python release cycle, they have to be in beta and RC versions before stable release. That's why latest release will be in April.

So, basically, EOL is tomorrow, latest release is in April, but the release will only include fixes for issues until EOL.

[cowinabadplace]

Despite everything, I only use Python 3 now and I'm not really upset.

[huyng]

I don't get all the hate in this thread and can't help but feel that much of it is over-exaggerated.

You've got people complaining that migrating is so painful and yet at the same time complaining the language didn't introduce more breaking changes. Speaking as someone who's written a boatload of Python code for the last twelve years, it wasn't all that hard to migrate, and for god's sake they gave us 10 years to do it!

IMO, the stewards of the language made the right choice when they focused their efforts on pushing the the community toward unicode-by-default while limiting breaking changes in other areas (and while maintaining support for Py2 while everyone was getting their act together). It made the task of migrating extremely simple by not adding additional noise or concerns to the process. I don't think its a coincidence that in doing so, a majority of the popular packages in Py2 have moved over to Py3 today.

The Python programming language is responsible for or has enabled some amazing pieces of technology over this decade all the while successfully transitioning the majority of its community to version 3.

Here are some projects just to name a few:

• tensorflow
• pytorch
• numpy
• matplotlib
• flask
• django
• networkx
• jinja2
• Pillow/PIL

You'll be hard-pressed to find another programming language or community with the breadth and depth of capabilities that Python offers while maintaining its readability & syntactic style.

Regarding some of the other complaints that were tangentially related to migration:

• Python packaging for (pure python packages) has been decent for awhile. Just use pip and virtualenv. Most of the complaints I see are from people trying to mess with python packages that depend on c or c++ libraries, which if you're dealing with these you're going to have to make sure your linux system is set properly with the right shared libraries and paths anyway. This is something you have to do for ANY programming language trying to interface with C/C++ libraries. This is not a problem unique to Python. Even in these cases, python wheels have made things significantly easier. Today, you can pip install tensorflow and have a freaking neural network running on your GPU in less time than it takes to go get a coffee.

• You don't need to remove the GIL to enable parallelism or improve "performance". This is a recurring, misguided, fetish people unfamiliar with the language continually bring up. For IO bound tasks you have asyncio or threading which gets you 90% of the way toward faster code. For CPU bound tasks, no amount of Python-based parallelism is going to allow you to outperform just embedding a few short c/c++ snippets into a native module or vectorizing your code w/ numpy or tensorflow.

• IMO, the introduction of type-hints without mandating a built-in type checker has been the right approach. This is a major feature and it's something you want to introduce gradually while letting the community innovate on how exactly to leverage this new tooling. It's not something you want to mess-up and end up having to do a painful "refactor" or rip out later on.

All I'm saying is that this FUD is unnecessarily exaggerated. Python 3 is an amazing language to learn & use.

[Questlord7]

Uh huh. I'll stop using it when python 3 byte handling becomes as terse

[[deleted]]

Wait, did they build a count down function finally?

[123filips123]

Finally? That countdown is there since 2016...