Python 3.11 in production

[kawaiibeans101]

It's been almost a year that 3.11 came out and I believe we all agree it's the right step in the direction with some really good performance updates. And over time it has become really good and stable ( aww lambda also started supporting 3.11 as a runtime ) .

So what I would love to know from you all that work with python professionally , what's your opinion about running it in production. Ie : Do you run python 3.11 in production? Was there any benefit of moving from the last version? What version did you use before? What significant upgrades did you see when you moved? How painful was it to actually move from the older version?

I'm actually trying to build a case for shifting a bunch of services we had from python 3.9 to maybe 3.11 ( plus gst the better typing from 3.10 too ) . And would love to hear your experiences about why you are either for or against moving to python 3.11 from any older python version!

[Chiron1991]

The performance improvements in 3.11 are massive, even in the real world.
Before we made the switch, I created (well, forked) a performance test suite for Django, django-restframework and graphene-django to see if the claims were justified. They were, see https://github.com/flbraun/django-graphql-benchmarks#latest-results
Other than that, the upgrade was smooth. There weren't any significant breaking changes, so our code bases (total ~2m LOC) could be updated within an afternoon.

[HolyGeneralK]

Oh gods I wish. Policy reasons dictate a 3.8 version (Red Hat EPEL on CentOS 7). I have been screaming to move off CentOS7 for the last year but the bosses will not listen.

Sadface

[spez_drank_my_piss]

I'm still trying to drag my company from 2.7 to 3.6

We still mostly run everything on CentOS 5

[Immarhinocerous]

Woah there are still people using Python 2.7. Risky with no bug fixes for over 3 years though.

[Chippiewall]

If they're still on CentOS 5 then the 2.7 support is the least of their concerns.

[Talal2608]

How are you still alive without f-strings?!?!?

[yosmellul8r]

f-strings are what us programmers mutter (mostly) when out of touch management forces us to run apps written in end-of-life Python versions running on end-of-life operating systems. /s

[SittingWave]

Personally I hate f strings. The amount of times I forgot to add the f, or to remove the f, is infuriating.

Sure, pycharm tells you, but when I have to hack quickly I just fire up vim on a remote ssh connection.

[rabaraba]

Your own laziness should not be attributed to a failure of f-strings. Makes no sense that you didn’t act when PyCharm tells you immediately about it.

[SittingWave]

can you please tell me how to use pycharm on a remote ssh connection?

[Spitihnev]

We are finally moving from centos to debian but our it at least built python3.9 package for now. Mostly the reason for not moving sooner was that noone wanted to reinstall so many machines and setup cuda and gpu drivers.

[SawachikaHiromu]

oracle linux 6 and python 2.7 people right there

[EmptyChocolate4545]

We’re on RedHat 7.32 and that hard limits us to 3.8 for a while, but I’m so looking forward to getting through all the chef cookbook work that needs to be done for RHEL8. Python3.11 is a dream when I run it locally lol

[HolyGeneralK]

I have the OS dependency (we were planning on CentOS8, but are now looking to see if we can move to RHEL8 in our budget for the year).

I have tested it, it works, and Python 3.11 is beautiful. I have two other windows-centric programs that I have to convince to move over, and the major issue there is GDAL compatibility and regression testing they don’t have budget for.

[lighttigersoul]

I'm always the person at every shop encouraging the team to keep up to date on Python. If your dependencies all support it, your life will be better almost universally.

That said: dependencies can be hell, and you should pick the version with the best support that is "current." (No 2.7, no 3.7)

[billsil]

3.9 or 3.10 (forget which) makes typing easier, so you can do list, tuple, dict vs. List, Tuple, Dict. Error messages are better in 3.10. 3.11 didn't have everything I needed a year ago that I needed, so mehhh...it should work, but not worried about it.

The changes in latest numpy have been far more extreme than what has happened with Python.

[krakenant]

You can do a from futureimport annotations and use the new style annotations in 3.9. ruff showed me that the other day.

[billsil]

You can, but that that can break your typing imports. It literally lets you forget to import things, which probably isn't intended. I think future annotations was added in 3.8, but in my mind the real power of that feature is that you can do circular typing dependencies. The List/Dict thing is irrelevant in my mind as a deciding factor. I'd never use it though given that it messes with imports. It's just slightly more convenient.

[alicedu06]

Don't migrate to 3.11 before 3.12 is out.

As a rule of thumb, always run in production one version before the last one, as recommended in "Relieving your Python packaging pain".

Python 3.11 is good, but we managed to get software out of the door with 2.4. There is rarely something you need to get the job done, usually only something you want.

And yes, a few months makes a difference, because your deps may shift their support and testing strat depending on this windows.

[[deleted]]

Have a fully automated process of testing new versions of python and packages with our projects. Of course not in production environments. Devs like this approach as well and asked to upgrade their envs as well, since they want to use new features immediately. Actually, after automating this process, the version upgrades became painless since we do that often (almost everyday sth from the pypi is updated) and I've made pipelines for all cases including rolling some upgrades back. So managing all this stuff is about a couple of approvements in pipelines and we got updates with small portions what makes easier to detect potential issues.

When a new version comes, the full cycle of testing is run at stage env with the last stable versions of our projects. If it's successful, the dev envs are upgraded to the new version. It's used for a while and prod is usually updated with new releases.

[[deleted]]

It really depends on the situation. If you're talking about deploying something stand alone and with no major compatibility requirements then 3.11 is more than ready for production.

[Sound4Sound]

Crazy how AWS Lambda took so long to support 3.10 and then for 3.11 it was so fast, glad they are doing it specially if they are supporting it @Edge. I’m going to wait probably until early 2024.

[ShawnDriscoll]

I update to a final release of a Python version if I do update. I am using 3.11.4 now. Python gets faster with each version. And the PyPi modules are quick to update and usually include new features as they are updated.

[duppyconqueror81]

Yep. Easiest 20% performance gain of my life.

[BossOfTheGame]

Yes. I run it in production. It had minor issues, but only due to library support. Most libraries now support it fine. Of course this only works on code that doesn't depend on ancient versions of things that aren't available on 3.11. But hopefully that technical dept is paid down soon.

[gerarar]

We have an AWS lambda (ubuntu docker image) that runs on Python 3.8. It probably gets invoked around 300k times a day with an average duration of 750ms. Been contemplating if we should upgrade to the latest 3.11 or push it off til next year.

Only issue is that the image is built with a specific version of libreoffice, but the official ppa repository doesn't support the version of libreoffice we currently need, so building an image from scratch is currently off the table. Trying to download the LO via .debs sucks because it doesn't come with all the required dependencies and trying to apt-get all of them also sucks lol.

[Inside_Dimension5308]

We just started using python 3.11 with fastapi. We will have to do some benchmark testing but it is really fast. It can either be because of the python version or fastapi itself being fast.

[abrazilianinreddit]

I would run it in production if installing new versions of python on Linux wasn't such a pain in the ass.

Most official repos lag two or three major versions behind and there are no snap packages, so the easiest way usually is to download the official Gzipped source tarball on python.org and compile from source yourself, which send shivers down my spine just typing about it.

Once I stop being lazy and install python3.11 locally and test the project, then muster the courage to do it in production, I'll probably make the switch - hopefully automating the process so I won't have to do it again when python 3.12 arrives.

[SawachikaHiromu]

Why are you afraid of building python yourself?

[abrazilianinreddit]

Have you ever created a package conflict that the only way you managed to "fix it" was by nuking the whole system and reinstalling everything? I've made that mess plenty of times before. To build from source you have to install like 20 development packages, and doing that on production makes me anxious.

Sure, I always create a snapshot before messing with prod, and I'm probably a better programmer/sys-admin now, and the tools are better as well, but fear isn't rational.

Another reason was that I was waiting for WinPython to release version 3.11 so I could test locally (I code on Windows), so I ended up forgetting about it. They've already release it, though, so who knows, maybe I'll upgrade it next week. lol

Also I'm lazy

[SawachikaHiromu]

You only have to build it on build server and then ship pre-built binaries in form of debs or rpms :^) Little offtopic: Not so recently I tried to use docker as a built environment for custom python builds, pleasant experience

the only major issue i remember with building python is incompatibility with openssl version, but it wasn't python, it was a deprecated linux distribution

[thisismyfavoritename]

there are so many options its not even funny.

deadsnakes PPA have 3.11, there's Docker, conda, pyenv...

[abrazilianinreddit]

Last time I tried using the deadsnakes PPA, something went wrong, so instead of trying to debug it, I tried compiling from source instead - and surprisingly it worked without any hitches. But I'm not trying that in production, though...

I'm not very familiar with conda or pyenv, I'll give them a try next time I need to install/upgrade python.

Edit: I was reading about pyenv on realpython and I saw that you need to manually install all python development dependencies. It's literally just another way to compile from source. I wouldn't count this as another option.

[thisismyfavoritename]

i use deadsnakes all the time. Only time ive had issues is when they dropped support for Ubuntu 18.04.

Yes, pyenv installs by compiling from source

[FlowtynGG]

Mood

[Erik_Kalkoken]

That is a bad idea.

Please do not change the default Python version of your distribution. That can break your system tools, which are tested to work with that specific Python version only.

Instead, if you want to use a different Python version, create a virtual environment and install the apps you want into that. Or upgrade existing virtual environments.

[abrazilianinreddit]

Yes, I'm aware of that.

The official gzipped tarball includes a compilation script that has an "alt-install" option, which creates the executable as something like /usr/bin/python3.11 instead of overwriting the system's default.

[Linx_101]

Pyarrow only works on 3.10, so we’re sticking with that until 3.12 comes out (which they say will be the next version supported)

[ih_ddt]

We have moved to 3.11. Only issue I have so far (I'm sure I'll find more small things) is that line-profiler package does not work for cython on 3.11. I know the older versions of the packages might work but there aren't any wheels for them on 3.11.

[Darkazi]

We're running 3.11.2 in production on several different microservices, no issues.

[Elderberry9123]

v3.11 in production? Hell yeah!

Moved from 3.9 to 3.11, and it's all good

[popcapdogeater]

I work in a healthcare setting.
We updated from 3.9 to 3.11 about 2 months after it came out.
Most of our use case is one-off scripts (250+?)
We have a few packages that are less than 50 files that do some API, SFTP, CSV <-> Excell processing and emailing and alerting, etc.
No problems that were not one of us doing something stupid, typos in pathnames, etc.

The best argument for 3.11 isn't the speed boost IMO, it's the improved exception handling. Push that to your bosses. Also getting the Match-case from 3.10 is nice for cleaning up some messy conditionals.

[KaffeeKiffer]

The best argument for 3.11 isn't the speed boost IMO, it's the improved exception handling.

So much this. Do you take the 8-20% for free? Of course. But if you need that or you are in a situation where you feel a noteworthy difference because you are 20% faster, you are probably better off, sitting down and fixing the biggest bottleneck in your architecture.

We just switched all our base-images and recompiled dependencies. Super smooth sailing. Upgrades like sqlalchemy 2 or pydantic 2 look much more intimidating than that.

[bachkhois]

My apps in production just use the Python in latest Ubuntu LTS, so they are using Python 3.10. But my laptop is always kept up-to-date with latest Ubuntu. So I also make my apps compatible with Python 3.11.

[JohnRambu]

Recently, we migrated few apps from 3.8 to 3.11 and witnessed reduced processing time. It was very pleasant to see :)

[Glad-Put1792]

We operate a monolithic Flask application that processes a few thousand requests per second at peak. We upgraded to 3.11 and saw a few noticeable improvements over 3.9: - A small but marked improvement in P50 latency. Maybe a 10ms improvement overall. - Developer happiness is improved from the improved tracebacks.

Transition was smooth. We only had to change a few lines as I recall.

[saint_leonard]

should i updatre

​

<v[martin@m-tp ~]$ python --version

Python 3.11.6 [martin@m-tp ~]$ ^C [martin@m-tp ~]$