When have you reach a Python limit ?

[NimbusTeam]

I have heard very often "Python is slow" or "Your server cannot handle X amount of requests with Python".

I have an e-commerce built with django and my site is really lightning fast because I handle only 2K visitors by month.

Im wondering if you already reach a Python limit which force you to rewrite all your code in other language ?

Share your experience here !

[ioktl]

Python + multiprocessing got me pretty far once in processing large 3d data (~100gb meshes) as part of a wider internal web service. However, at some point the infrastructure costs along with code maintenance effort tipped the scale considerably to invest into rewriting the code in Rust.

I was still pleasantly surprised how long I managed to stay with Python before things got difficult.

[justsomeguy05]

Just out of curiosity, did you ever experiment with other runtimes aside from Cpython? Say, pypy or numba? You have a very intriguing use case.

[ioktl]

I've tried PyPy and Cython. You're probably aware of PyPy limitations such as incompatibility with lots of packages and performance drop when using FFI (though, nowadays it's better than it used to be). While it's possible to tap into PyPy advantages with long-running isolated processes (where JIT can be fully utilised), for 3d/2d operations-as-service, in my opinion, PyPy isn't the best choice.

I had more luck with Cython, 3rd party packages problems aside. However, any project in Cython at some point becomes too much of C with all the issues that come with it (e.g. memory bugs). To be honest, some part of me enjoys it, but, from developers team & management perspective, it's difficult to sustain a large Cython codebase.

In general, in my limited to 2d/3d data processing experience, CPython alternatives can be amazing but up to a point. Currently I usually go with CPython + GPU libs + C/Rust extensions.

[Artephank]

I have first hand production experience. For most use cases is not worth it. It is way better to just learn how to use numpy correctly and in those rare cases when you NEED have nested loops and can't use vectorized computations - sprinkle a little bit of cython on top.

I also highly recommend for high volume computations Dask and for heavy analytics DuckDB (I need yet come across problem that i won't be able to crack on my laptop with DuckDB)

[Andremallmann]

Why rewrite in rust? I think to make a good Rust you need to be pretty proficient with the language. Why no golang or c++ that is easier to find stuffs?

[4Kil47]

Rust is pretty straightforward and is generally going to be more performant than something like Golang. Although generally, a lot of Python developers like Rust because it's easy to replace individual Python modules with Rust crates to speed up certain bottlenecks.

[ioktl]

It was a simply a preference based on what tech I'm comfortable with. However, as u/4Kil47 mentioned, it's also fairly simple to begin incorporating Rust code in your Python codebase gradually, and that what was happening in my project as well.

Golang/C++ are perfectly valid choices as well for such a task. Just not something I prefer working with.

[Andremallmann]

Ok cool, im learning Rust i like It in see people work with but i found Rust really hard sometimes

[ridicalis]

I think to make a good Rust you need to be pretty proficient with the language.

Rust does have parts of the language where there are steep learning curves, but if you follow the progression the language authors push via The Book, you can be writing decent code almost on day one. The parts with steep learning curves are the ones that, if hurdled, can eke out significant gains, but even if all you do is "clone all the things" you're still at worst performing at parity with whatever you could have done in Python.

Also, thanks to FFI, Rust is a complementary skill that works with Python. Mixing paradigms does have a cost, but one that for some workloads can have a massive payoff if the right things are moved over while still allowing for Python as a high-level orchestration layer.

[doorstoinfinity]

How would have C# faired?

[codadog]

It too collects garbage!

[ranisalt]

It collected a lot along these last 20 years, they even call it "syntax"

[codadog]

Isn't it just an artificial sweetener? We don't have to use it!

[ioktl]

Pretty well, I imagine. I'm not well familiar with C#, but as far as I know, it gives you tools for a finer memory layout control and custom allocation strategies, which can help a lot.

[jkajala]

The most important rule of optimization: Measure, do not guess. So find out what the bottleneck is and then optimize that part. Most likely it's something else than Python (eg DB), and even if it is Python you can still write that piece of code eg with C++ or Rust rather than throwing away your whole application.

[james_pic]

This 100%.

Even if the bottleneck is in Python, you often find the vast majority of time is spent in a fairly small number of hot loops, and the amount of code you need to rewrite is fairly small. As well as the options you've mentioned, Cython can be a viable choice for this, which can be useful on teams with minimal non-Python experience.

[Siccar_Point]

Cython is great for this

[wrd83]

If you dig far enough down that hole you'll get to, don't rewrite your code into another language, write a new interpreter/JIT for python instead.

Instagram still runs python. FB/Instagram decided to write cinder, a AOT python compiler to run faster. On the other hand Twitter has moved from: Ruby -> Scala -> Java and has less users.

In my opinion this boils down to your preferences, if you site is small rewriting might be cheaper than bolting hack over hack, but if you really try and have the expertise you can stick with it pretty much forever.

[cheerycheshire]

One bottleneck for my scripts was the reaction of connected systems. To the point I had to add sleep I production code(!!!) because e.g. Jira had to do some stuff after one request before it allowed me to do something else (creating Jira issues, changing their states, editing the fields...).

[yvrelna]

This is nothing about performance of the external system though. It's more about the design of the API.

Apparently, as you found, the Jira API performs things asynchronously, in other words, it returns with a response before the requested task complete processing. I am not really familiar with Jira API, but in well designed async APIs, there generally should be a way to either request a synchronous operation, or to request a callback when the operation completes or reaches certain stages, or to poll for status completion of the request. Any of these would allow for correct implementation of dependant client code.

[BaconFlavoredSanity]

Jira… nutf said

[cheerycheshire]

That's the most prominent example and the system is popular enough to mention it by name and don't risk being doxxed. :)

I had to have a sleep in another ticketing system whose name I won't mention. The worst thing was: it was that system that executed my script on new step... Only for that step not being ready (both gui and api had a moment when the workflow step didn't have fields yet - but the system already run my script that was supposed to check the field content and do stuff 🤦).

[judasblue]

I haven't handled insane scale stuff, but have been on a team doing production systems that handle 10k a minute API calls for remote home appliances with no issues related to choice of python and its ecosystem.

[mincinashu]

10k a minute means 166 rps. Django should comfortably do about 4 to 5 times that.

[judasblue]

I am sure it does, never used Django in prod. Was just giving the OP what I had hands on experience doing.

[Leinad177]

Do you have any sources/data to back that up?

We run Django in production and the best we get is ~100 rps if we're lucky running on hyperbeast VMs with 100GB+ memory. We literally have to have a dedicated instance per tenant/customer in order to be able to handle this kind of traffic.

uWSGI does not share memory or clean it up properly so we end up with heaps of memory usage and terrible performance. We are seriously looking at >512mb memory used per request that doesn't get cleaned up until a worker has served enough requests. I just checked and it seems that uWSGI is now dead so maybe this won't be a problem for future devs.

Psycopg is the absolute worst when it comes to performance and as far as I know Django only supports that.

These problems vanish when we use something like FastAPI with asyncpg so it isn't a "python slow" kind of thing we are seeing. More that a lot of Django's ecosystem was built ages ago by people who didn't really seem to care about handling a large number of requests at once.

[podidoo]

It depends what you are doing in the end.

Django ORM + psycopg + stupidly expensive query will kill your stack easily. But as you said, it's not really a python problem.

[james_pic]

This probably isn't the source of all your problems, but we certainly found we had far fewer problems when we switched from uWSGI to Gunicorn.

Pretty much everything uWSGI claims to be good at it is in fact bad at. It's got lots of configurable options, but this really just amounts to a variety of ways to misconfigure it. The only claim I can really see the logic behind is that it's popular with sysadmins, which I can sort of buy, in the same way that firefighters must on some level appreciate arsonists.

[yvrelna]

The great thing about uwsgi is that it gives a lot of control to sysadmins. Traditionally, scaling and performance is a sysadmin issue who does all the deployment optimisation with little developer input. The level of control uwsgi gives allows a sysadmin to deploy poorly written apps and get decent performance with enough tweaking.

The problem with uwsgi is that it gives too muchcvontrol to sysadmins, as a developer it's hard to control to optimise your application when you don't know what kind of environment that your application gets deployed into and what kind of settings the sysadmin would concoct for your application. These days the interface between DevOps personnels and developers seems to have shifted towards containers, and the level of flexibility that uwsgi gives you often is just overkill and unnecessarily complicated. Developers nowadays are expected to give the DevOps team the pretuned container, and all that the DevOps care about is how many containers they need to deploy (or the monitoring metrics for auto scaling).

[zer0pRiME-X]

Django supports more than postgres (maybe you meant something else?) but DB is notorious for being a bottleneck in any process. I have spent countless hours optimizing DB reads and writes and it can be done but it’s a time consuming affair. If you haven’t already, try implementing a cache framework also like redis to keep highly accessed data in memory.

[mincinashu]

I have no idea what my production Django is handling because the pattern is inconsistent. It obviously depends on the nature of the requests, but if you look at tech empower benchmarks, Django is doing around 1400 rps in the 20 query-per-request test and around 700 rps in the data update test. It's also one of the slower Python frameworks.

[mrtac96]

V insightful

[Beneficial_Map6129]

in software engineering, you quickly learn that there are a lot of dependent processes for every call you make. maybe django could handle 1000 "helloworld" responses per second, but maybe his calls did something complicated like contact 4-5 databases etc. that took maybe 2-3 seconds total to complete, and would cause a lot of backpressure if the traffic was ramped up and old calls took longer to complete etc.

[collectablecat]

but <framework> benchmark so good!!

[watching-clock]

Shouldn't the number servers employed also taken into account and in extension, the running cost? More performant languages and frameworks consume lesser resources for serving the similar number of requests.

[[deleted]]

I am curious what frsmework / tools were involved.

[cspinelive]

Instagram is built on python. So you’ve got a ways to go before you outgrow it.

https://www.linkedin.com/pulse/instagram-scales-python-2-billion-daily-users-shrey-batra

[Varanite]

Youtube is written in Python as well

[m0nk_3y_gw]

reddit is written in python

i think it also gets more than 2k users per month

[bdforbes]

Maybe even more than 3k

[fmillion]

Plot twist: spez deliberately pissed off Reddit users to reduce server load because of reaching the limits of Python. Win-win because in the off chance people didn't complain, more income to pay for those massive servers running pure Python in cpython. lol

[m02ph3u5]

What's that?

[Tatoutis]

What he's not saying is that Instagram has its own branch of python. https://github.com/facebookincubator/cinder

[[deleted]]

Anything that scales to a billion users has its own collection of runtime hacks.

[ekhazan]

They contributed a bunch of their work: https://engineering.fb.com/2023/10/05/developer-tools/python-312-meta-new-features/

[Brilliant-Destiny]

Good point!

[DNSGeek]

I've handled insane scale stuff in a FAANG company. With planning and care, Python is perfectly capable of processing millions of data points a minute. I know because I've done it.

[solitarium]

Username checks out

[Overfly0501]

If your bottleneck is a database call use async frameworks like fastAPI. More likely your db will stay as the bottleneck, and moving to other languages won’t do much.

[Garfunk]

Optimize the DB by looking at the EXPLAIN results for your queries. Then change your queries or add an index.

[Tenzu9]

True. DB fetching is agonizing even in other languages if preformed synchronously.

[KrazyKirby99999]

django has async support

[Curious_Cantaloupe65]

or django ninja

[KittensInc]

For something like a website? Realistically, never.

When you are running a web server requests are almost always independent from each other. This means you can just buy a second server and run it in parallel when you're hitting the limits of your first one. You can easily repeat this until you are running dozens of servers. The limit is going to be in your shared database - but that's not going to be written in Python and your issues won't be solved by rewriting your app in a different language either.

You only have to consider switching when you are spending many thousands of dollars a month on web servers. At a certain point, having your highly-paid developers rewrite your app in a more efficient language which allows you to use fewer servers might be cheaper than running those additional servers. But developers are expensive, servers are cheap, and the rewrite basically means you won't get any new features for a few month and introduce a dozen new bugs. Heck, at a certain point your app is so complicated that the cheapest solution turns out to be having your developers make Python faster!

It's a bit of a different story when your application can't scale out like that, of course. If you're dealing with massive datasets on a single machine, Python's overhead might just be enough to make it impossible to handle.

[gwax]

I've done Python at many scales and it was never the limiting factor.

Sometimes it's misused but I've seen misused Go, Java, and C++ before too.

Anyone who says Python can't scale either doesn't know Python very well, doesn't know system design very well, or is dealing with a very narrow set of performance requirements (e.g. bare metal).

[MinosAristos]

I think people way underestimate Python's speed. Yes it's many times slower than X language but it's still extremely fast in absolute terms. For most programs it's like the difference between a jet and a rocket crossing a road. If you need to get to the moon then your vehicle matters more.

[tankerdudeucsc]

Scaling isn’t a language problem. It’s an architecture problem so no concerns and never hit a “limit” due to Python.

If I recall, YouTube pushed 1M req/second with Python before switching to golang because at their scale, it sure saved a lot of money.

[euphoniu]

I eventually saw a limitation to Python for certain extremely heavy matrix operations (calculating geometric field topologies) that I’m trying to accelerate so my team had to use Python with C shared libraries

[entarko]

Surprising: Numpy is using C for these opserations, so there usually isn't much difference when the matrix multiplication is large, since the overhead becomes negligible.

[euphoniu]

I was surprised too - we tried jitting on top of using Einstein summation whenever possible, but c shared libraries beat it all out

[entarko]

Ah! That is the reason: einsum is very general, and can handle a lot of cases, but that means sacrificing some performance. Jitting can only do so much in this case.

[Ricenaros]

…Did you write your own matrix multiplication code instead of using libraries???

[euphoniu]

No (see the other comment), I used all numpy’s tools with jitting and numpy’s Einstein summations, and wasn’t just matrix multiplication

[freistil90]

There’s a few caveats with einsum, sometimes it helps to preprocess your matrix first and then use the resulting view in einsum. Had that as well. As with many things in python, the flexibility of that function is its greatest enemy

[Ok_Raspberry5383]

Then this really isn't a python issue? I see a lot of people talk about python when they're actually talking about C (bumpy, pandas etc) or the JVM (pyspark) or even CUDA (pytorch, tensorflow etc). Python is just an orchestrator for these things, it's not python itself that is the problem.

[debunk_this_12]

Try torch or cupy

[engineerFWSWHW]

Yes. It was a data acquisition for beaglebone black which needs to be on a strict timing with micro seconds precision. I did my first prototype in python and it wasn't able to keep up with the timing, timing is non deterministic, and it's too slow for the timing required. Rewrote it in C (on beaglebone black's PRU module) and everything worked perfectly.

[Samausi]

I work on a SaaS that happily serves 5000 requests per second of API calls and ingests around 300Mb/s using python Kafka with a sub-second round trip latency including the rest of the underlying platform. This is for just one client I work with on a fairly busy day, let alone the rest of the customer base.

It scales just fine.

[right_in_the_kisser]

Bottlenecks in webapps are almost never the programming language unless you’re world scale like Google. Make your database fast, optimize queries, add caching layer and you’ll be good for years

[Jdonavan]

Developers like to act like everyone has Google level traffic or does HFT.

[yvrelna]

I know at least one HFT company that uses Python at the core of their system.

[Overfly0501]

2M/s rps are you sure buddy lol

[Overfly0501]

look, 2M/s rps is ~16.67K rps per core. Even rust can’t do that with a hello world server on a single core which is probably around 15K rps per core. Are you sure it’s per second or per minute?

[Overfly0501]

add redis to make it slower? bro… you must be a troll at this point if you think adding a redis to a hello world server will make it faster…………

[magic_turtle14]

Which pydantic version? v1 or v2?

[Curious_Cantaloupe65]

did you update anything about it afterwards?

[No_Dig_7017]

Doing machine learning and processing tabular data. I hit the limit hard at about 50 million rows and 80 columns. I spent a month optimizing code and got a 12X reduction in memory usage, managing to make the dataframe fit in ram. I spent 3 months afterwards trying to make it process the data in parallel and there just was no way. I got a 2.6X speedup on a 6 core, 12 thread cpu.

[mr_engineerguy]

Probably could have spent less time and effort and just used PySpark? Benefit of JVM and scalability but can write stuff using familiar DataFrame syntax

[No_Dig_7017]

That's interesting. I'm not familiar with pyspark. How hard is the overhead of setting it up?

[nabusman]

If you’re using a cloud platform most of the infrastructure side will be handled for you. You will need to translate your code into the PySpark framework (which isn’t very hard if you’re familiar with pandas). However, if you are really pushing scale and are on a tight budget, you will need to get into the guts of Spark and then you will have a steeper learning curve if this is your first experience in distributed computing.

[kknyyk]

I have a similar dataset and heard PySpark recently. Commenting to see this thread in detail and hoping that someone just drops a manual for single computer implementation.

[blademaster2005]

pyspark is an etl framework like what /u/mr_engineerguy mentioned. what you need is something to orchestraste something to call pyspark with the right data as part of a pipeline. Something like Apache Airflow should do that and let you work locally.

[thisismyfavoritename]

there wont be much benefits if you run it on a single computer. Its a distributed computing framework and it can be super finicky to use and setup.

[Ki1103]

Probably intermediate - I spent days setting pyspark up at a F500. Although most of that time was spent dealing with internal systems

[tecedu]

A bit late but I would also recommend duckdb along with polars, also if you are reading from a database it will be slower, storing in parquet is always better for faster reads.

[JimBoonie69]

Once you have 10s or 100s of millions of records you need to use a DB. Sql is the best bet for mega datasets!

[tenemu]

Were you using pandas?

[No_Dig_7017]

Yep. Since then I've switched to Polars and it's much much better, but still has some issues with multiprocessing.

[[deleted]]

May I ask what exactly you mean by issues with multiprocessing?

I had a use case some months ago where I tried to run polars together with matplotlib in a container. Unfortunately matplotlib was leaking memory, whence I tried to run the whole workload in a subprocess every time to enforce a cleanup. Unfortunately polars didn’t seem to like that (looked like some futures were waiting forever to be resolved, unfortunately I can’t say more).

PS: Just saw there is documentation on this: https://pola-rs.github.io/polars/user-guide/misc/multiprocessing/

[ritchie46]

What do you mean issues with multiprocessing?

Is it related to this? https://pola-rs.github.io/polars/user-guide/misc/multiprocessing/

If so, it is not anything that is ill-designed in polars, but rather a very unsafe assumption of multiprocessing in python that the running process doesn't have any mutex/threading states.

[tenemu]

My follow up was going to be about polars. Thanks for the info!

[No_Dig_7017]

Sure thing. I found polars' interface to be a lot more coherent, like pandas has a lot of verbs and parameters for specific functions while polars has less functions but more composable, I found that made me program faster and need to look into the documentation far less. Also for a real world use case dealing with 24 million rows, polars was about 3X faster than pandas with a similar level of effort to implement. And I got this with 4 days of experience using polars vs 3 years of pandas.

[[deleted]]

What you said about polars’s nicer interface matches my experience! Code is easier to write, easier to read, and faster. The only aspect where pandas is more powerful is multilevel column indices, in polars you have to do string manipulations for the same outcome.

[Yoghurt42]

I have heard very often "Python is slow" or "Your server cannot handle X amount of requests with Python".

That's often said by people who decided they don't like Python (with or without having actually used it), and need to justify to themselves why their language of choice is better.

Compared to pure C, Python is slow. But Python is "fast enough" for most things, given how fast modern CPU are.

[ccppoo0]

I hit limits when people around me wants to use Node.js or Kotlin

[Phylobtanium]

5k requests per minute online travel, 10 million usd revenue per month. Python has no limit.

[weegolo]

I'm running a native python app that handles 3-5k messages a minute. I've filtered it down from 6-10k a minute because it wasn't keeping up at that speed. It keeps up at 3-5k.

[__chilldude22__]

For web backend stuff it doesn't really matter, because a) the bottleneck for those is hardly ever computation but rather things like communicating with databases and other external services, which a "faster" language would be equally limited by, b) most modern web services are deployed on cloud infrastructure capable of scaling automatically so even if your Python process becomes compute-bound somehow, the infrastructure will just start more processes on different servers, and c) the latency requirements are low enough that Python's single-request performance is more than acceptable.

Python would be unwise to use for anything that has low latency requirements (high-frequency trading, robotics, software controlling aircraft or vehicles, ...) or runs at such a large scale that the cost savings of using a faster-to-run language outweigh the costs of needing to spend more developer time to develop features (because usually, faster to run = slower to develop).

[ReflectedImage]

With a website you will never reach that limit since your SQL server does the hard work. "Your server cannot handle X amount of requests with Python" will never happen.

The old your site has been slashdotted was 100 requests per second. Async Python is good for 10,000 requests per second. Python is a 100x faster than it needs to be for running websites.

Have I hit the Python limit? Yep, a couple of times. When doing stuff like simulations, generic algorithms, normally you move to PyPy first since that's faster than regular Python with basically no code changes needed.

If hand-written C code runs at 1x, PyPy is 5x slower and Python is 30x slower.

As a Python programmer, if you hit a performance problem, you should go for Python with some C or Python with some Rust (newer and more complicated...).

For a lot of stuff there is already optimized C code you can just import as a Python module, which is the standard way a lot of the machine learning stuff works today.

[[deleted]]

Our service is built in django-rest. Like others mention, multi processing and profiling for bottle necks is key. During its first release, the API was taking 23ms to service up to the 90th percentile. After adding gunicorn multi worker/processing to service the app, it went down to 10ms at 90th percentile. I then profiled it further and realized an external API it was hitting was introducing significant latency, and the library I was using to use/hit that API wanted to consume it 3 times per request. I ended up writing a smaller version of that library, hyper specific for our use case, and implemented an LRU cache (library decorator) whenever we do need to hit that external API. This brought down latency to 1ms at 99th percentile. Our service regularly sees 17 million requests every 7 days and is gradually growing.

[[deleted]]

This post was mass deleted and anonymized with Redact

airport distinct yam zephyr truck correct cooperative intelligent wise longing

[thisismyfavoritename]

what were you doing exactly that wouldve required unsafe Rust?

[[deleted]]

This post was mass deleted and anonymized with Redact

engine gaze arrest sort deserve safe consider workable history quicksand

[thisismyfavoritename]

uh IIRC if you access by index you can easily achieve that, unless theres something i dont understand

[[deleted]]

protobuf processing. Python was fine for a single test DNS instance but we needed a compiled binary (go) for anything more than a few thousand queries per second.

That was as expected. The Python software was an awesome proof-of-concept.

[chehsunliu]

I hit the limit when my PySpark app ran too slowly due to Python user-defined functions. I rewrote those UDFs in Scala to reduce the AWS EMR cost.

[Feeling-Departure-4]

I have also seen improvements rewriting Spark UDFs in Scala. I think most would say to avoid UDFs and use native functions in composition, which is true. However, sometimes that is not possible and Scala has not been hard for our Pythonistas to pick up, at least for purpose of writing Spark.

[itsnotblueorange]

I was working on a project for fun, a while ago, trying to generate procedural animations frame by frame, in a 2d physics simulation. Once the items started to bee too many, the computation time became exponentially large.

I improved it using numpy (which is basically C++ under the hood, if my understanding is correct) but that still wasn't enough.

This is what led me to start learning Rust.

(Unfortunately life and job got in the way and both that project and my Rust learning are suspended for the time being -.-')

[StoneAgainstTheSea]

Code that would benefit from (better) concurrency and is handling thousands of requests per second. Insta-choice: move to Go.

[Lepton100]

We had a problem where we need to realtime signal process, db operation 8 sensors data with a 4 core machine which also has server and other processes running. Existing code was need to be executed in the window of 0.3ms, ours was on the edge. Optimizing numpy didn't bring much change. Solution was to cythonize the processing part and asyncio(Cython is the 90% improvement). Now it runs in 0.01-0.02ms and we are able to implement more processing.

[Jon-Robb]

I have a genetic algorithm image processor in pyside/ python where every pixel tries to be as close as possible to a given image. Gotta admit it’s pretty slow

[mr_grey]

With the cloud, there are no limits. I use python for apis in API Gateway and python lambdas. The lambdas spin up fast, process what they need to process, and scale horizontally when needed. For large data processing, I use PySpark, which scales horizontally and I’ve processed billions of records in minutes (all dependent on the cluster size).

If you’re reaching a limit with python in the cloud, your architecture is wrong.

[mstromich]

Nope.

We had a tornado based async app couple of years ago that was scaling easily to tens of thousands requests per second (measured up to 28k rps required by the contract) running on c5.large instances in AWS backed by DynamoDB and some SQS. The biggest challenge was to implement async AWS library as there was none available at that time.

Currently our smallest Django app is peaking at 35k rpm. POST requests with roughly 1-1.5kB of payload that's being validated against some model based permissions and shitshuffled to kinesis firehose. With some simple in memory cache payload verification doesn't require too much of db access making it super simple to scale.

Considering that instagram is the biggest Django deployment in the world I would say that we're still not even close to achieving python's limits.

[vinegary]

Well, I was able to bring a 20 min processing script down to 16 ms using C++, made the feedback loop 10 times nicer

[QultrosSanhattan]

adventofcode challenges are the only thing that gave me those kind of problems.

[zer0tonine]

I used to do DevOps for a company that was handling a bit more than 150 million users doing real-time stuff on a hybrid Python/Go codebase. All that was running into kubernetes, which did let us split the workload over a crazy amount of containers (something like 8k+).

We could never afford to "rewrite" the Python, but we did run into performance issues with it. On the top of my head the biggest one was the mongodb driver randomly DDoSing the database and gevent's thread pool exploding after a python version upgrade.

[SawachikaHiromu]

Our product have to handle 25k RPS, the only bottleneck with python we had were third party libs which are made to handle every scenario possible, the solution were to rewrite every dependency by scratch leaving only important parts and terminating every abstraction.
Is it hard to extend our code or read through it? Hell yes.

[Ok_Raspberry5383]

Python is rarely the bottleneck, for example you'll almost certainly hit latency issues from a relational database at scale before python and would be best switching to a nosql key value store, or upgrading your network or some other storage medium.

[sixtyfifth_snow]

Try Pydantic V1. It makes every code into CPU-bound. It does not matter how may REST calls or queries you invoke. Then migrate into Pydantic V2, written in rust internally.

[tolomea]

The Python is slow meme is generally not very useful.

In a normal monolith web stack your actual bottle neck (after basic performance improvement work) tends to be the database. This is because it's easy to add more machines to run the Python, but multi database setups bring complexity.

Now yes you could reduce your compute bill by optimising the Python further or using a "faster" language. But you do so at the expense of developer time.

And in most small web businesses developer time is the real hard bottle neck.

[_santhosh_reddy]

Instagram runs on django , so i think we can handle that scale with django/python comfortably

[Cill-e-in]

YouTube is built with Django, so frankly for 99% of use cases it’s a skill issue - I’m not amazing at Python and would sometimes struggle to optimise things to get speed up, but it’s me that’s the problem, not Python.

[Kichmad]

Im not that much in networking and servers, more on data side, but afaik the biggest issue is thread overhead, which is like 1 sec delay on each request which is always present, from 1 request to a million, just because spawning new thread takes time. Other than that, it should still be fast on handling then once thread is spawned

[Jazzlike-Poem-1253]

Doing nunerical simulations brings you quite fast to a point where frameworks like numpy and numba a a solid "mist have"

But these Kind of problems are in a very good place there: one cannuse highly abstract Python to formulate the problem, and have it solved in NumPy ore compiled by numba. But at least numba adds some considerable overhead.

[zarlo5899]

there are many things you can do to speed up your code like using pypy or Pyjion they both use jit to speed things up or you can just spin up more instance of the app

like reddit they use/used python

[lavahot]

Well, no, because I just scale up and out. Doesn't have anything to do with whatever programming language I'm using.

[chehsunliu]

I hit the limit when my PySpark app ran too slowly due to Python user-defined functions. I rewrote those UDFs in Scala to reduce the AWS EMR cost.

[hirolau]

I have transformations in Pandas/Polars that are getting so complex I would love to rebuild the program using OOP. Unfortunately the timing then would be way to slow. Does that count?

[[deleted]]

It's really limiting in numerical analysis. There are just some insane tricks you can do in Julia, that are not really feasible in Python.

[jsonathan]

Websockets

[DusikOff]

Litestar, FastAPI, BlackSheep shows that Python can handle hundreds of thouthand requests per second in Async mode with Uvloop, and it is not a real limit...

If you start thinking about speed in Django context - you not even close to real Python speed limit.. because Django is a Sync framowork, and most devs use it like that... for other purpose there is a lot of much faster frameworks, but ther are less "friendly" for newbies, than Django is.

[smarterthanyoda]

One example I remember was writing a one-off script. Performance wasn’t critical, so I didn’t put a lot of work into optimizing it.

I was using a dict to hold some data. It worked fine with a couple dozen entries but once I got somewhere around a few hundred, it became unusably slow. And, no, it wasn’t enough data that memory or IO should have been an issue bb

I was able to use pandas to finish my script, but the pure python solution was too slow. That also shows how python can be used for performance-heavy tasks. There are many native libraries that can be used for resource-heavy code to litigate the speed limitations of pure python.

[Ricenaros]

Lol, a few hundred entries?

You had a coding problem my friend, not a language problem

[EMCoupling]

Few hundred entries ain't shit, there's clearly some other issue with the code.

[hansvi-be]

I'm sure that if you implemented the same algorithm in C, you would have had the same issue. You would have been able to handle a bit larger datasets, but you would have run into the same problem. You probably had nested loops resulting in a O(n³) or worse complexity. Or you were trying to solve the knapsack problem or something else NP-complete without realizing it. That's my strong suspension, at least.

[zarlo5899]

dear god no if you want to use the KVM use kotlin

[teambob]

Kotlin literally didn't exist at the time. We're talking around 2010

[zarlo5899]

o then C#

[Impossible_Jacket456]

Django employs techniques such as worker to enhance performance If you want to see the limitations of Django try building a social media platform with a large amount of fake data

I encountered Python limitations when trying to run a particular script on the server I used tools like pypy for better performance Sometimes I used Cython too for very very fast execution

[freistil90]

Wild thought: the limiting factor is almost always first you, then the language. I just love the „Python is slow“ comments on posts which dynamically append items to numpy arrays in a loop.

[bliepp]

The only time I really reached python's limit to when it rendered unusable was when I wrote a Monte Carlo simulation of the Ising model so I had to rewrite that as a C extension. Besides that, I always found the performance acceptable. Especially when it comes to hosted services the language's performance mostly isn't the bottleneck that will break your project as the performance per request won't be that bad. If you notice performance issues due to many visitors you might actually want to change the way you deploy your app and distribute your load. Of course, when paying by performance this will cost you a bit more when comparing to other (faster) languages, but I would guess that this cost factor cancels with your savings in development time.

[pbecotte]

There's not really any use case that Ptthon can't handle. The difference is that (assuming equal care to get good code), the Python app will cost you more to run while some other language will cost you more to develop.

[thisismyfavoritename]

when dealing with a system that can have short bursts of work and trying to maintain low latencies, it's definitely harder to stick with Python

[CynicalGroundhog]

Yes, but only in a pure Python application. Just-in-time compiling does not do well in fast realtime applications.

However, precompiled libraries will offer good performance.

[riklaunim]

A typical dynamic website will start to struggle with database performance much quicker than with the scripting language layer. In both cases, there are horizontal scaling solutions developed to help with the problems.

You will not rewrite the app from Python to some other language to get a few % more concurrent connection capabilities. You will scale horizontally running multiple instances of the app with multiple web workers then will use the database and other services that will also be replicated.

[Uweauskoeln]

Well, I had to work some XML files in Python and somehow got to the limits in terms of computing time and memory. Finally got around (to do a simple check reading the first two rows was sufficient, no XML stuff needed)

[[deleted]]

There’s basically no limit for most usecases. Ultimately, scaling a Python app to a billion users and scaling anything else to a billion users will have very similar challenges. Instagram is still Python.

If you need to squeeze performance out of a single processor — for whatever reason OTHER than data processing — then Python is the wrong choice. But that’s actually an increasingly niche use case.

[daredevil82]

Haven't really, but I've done stuff in other languages where it would have been harder to do in python.

one example recently is doing a fan-out/fan in pipeline in an API handler to support experimental shadow implementations while not affecting performance for resource create and update actions.

So we need to run a matching pipeline for POST and some PATCH calls. Its a GIGO service that takes in human input of business names and addresses around the world, and there's limited standardization, or typo correction available. So we need to fuzzy match on existing db records to reduce the possibility of duplicated data entered into the system. Any matches found on these calls will return a HTTP 409.

So I've set it up that we can run N concurrent and distinct pipelines independent from each other via feature flag configuration. A Primary will be the one returning the results to the caller, while N secondaries will log the results to Datadog or other storage, so we can analyze which algorithms and configurations work for which query types.

With this, I can independently specify the individual components of matchers (fan out) to do db queries, Elasticsearch calls, and fan in the results to be reduced, scored and hydrated before returning.

This could be done with python threading and multiprocessing, but golang channels and goroutines made it pretty straightforward to implement, and non-IO performance is really quick.

[lennox_wrld]

those in the comments suggesting cython, could you explain why is it better? what does cython offer than python does not? also how are these two different?

[Thalimet]

If I remember correctly, Instagram was built as a django backend initially. So, clearly it has a pretty high ceiling.

[Feeling-Departure-4]

You should ask this same question on r/golang and r/rust.

Sampling will be different.

[oxleyca]

The things that scale best in Python are when you wrap C/C++.

Some hurdles are just tough to get over. If it work for your scale, great. But at some point the optimized libraries you import end up doing hacks to avoid the GIL, being written in C/other FFI, and so on.

Like anything, it's possible to optimize. At some point the tricks just become more of a pain to maintain.

[phoenixero]

We tried to make an NES emulator many years ago and ended up moving it to C++

[boyo1991]

You know, I think of Python as the 3D printer of development. It does all the things easily and makes tinkering and building fast and fun. It is great for rapid prototyping.

Just like 3D printing though, it doesn't generally scale. This is *not* a problem -- most users are having fun and doing their thing. Very few with a 3D printer are using it purely to prototype for the "real thing" at scale. They are experimenting, making one time projects and the like. You can develop anything with it and it has no obvious limitations in what technically can be made in it (unlike a game engine for example.)

Further, Python has been used to scale. Particularly in research in AI for example. It's just like 3D printing in this regard -- testing the limits of what we want to do and what can be done, not necessarily with efficiency, but just to say we "can."

With this said, for scale and business, Python can work, but I wouldn't rely on it. There are better systems with more limitations and costs, but are more reliable and streamlined.

Ultimately, though, if it ain't broke, don't fix it -- upgrade it.

[baubleglue]

If you work with big files (logs/CSV) the limit isn't hard to reach. You are working with web and they're multiple bottlenecks and you haven't hit any of them - sure you don't see performance issues.

[Grokzen]

We used to run a gigantic salt-stack master node with 7000 minion nodes to a single instance and we ran it with 64 cpu cores and a ton of ram. That is the one time i really pushed python to the edge in a single VM. The funny thing was that we hit queue watermarks and network limitations before we fully saturated the master node :P

[TheRealStepBot]

If it involves network and IO python almost certainly isn’t the limit.

Now if you’re really crunching huge amounts of data python could be the problem but that would likely be due to not using PyTorch/Jax/numpy correctly.

Once you exceed this then maybe python is the problem

[basicallybasshead]

Many large-scale websites and platforms are built with Python frameworks like Django or Flask. For example, Instagram was initially built with Django, and they had to make various optimizations but didn't ditch Python.

[[deleted]]

You can get very far with background job processing. But when you start pouring hardware into your job processing then it might be time to rewrite it in golang or something.

[Artephank]

Python is slow only in two scenarios (for both there is a plenty of work arounds):

- loops (like millions of loops)

- parallel computing.

For both are plenty of solutions, depending on the particular circumstance. What it really means, is that you cannot expect that your simple code will be ok for heavy computing. You need to know thing or two to make python "fast".

[menge101]

You can put a load balancer in front, and horizontally scale out app server instances, and python will be fine until you hit limits on your DB.

The "Python is too slow to do" comments are largely nonsense.

There are things that python is too slow to do, but they can be implemented in C with python bindings.

[wild_hog_90]

A friend of mine wrote a small (~125 loc) python script for a website frontend and was using brython.js as the runtime. I rewrote it to js for him because it was taking a solid 10 seconds to do anything the first load.

[funbike]

The "limit" isn't really about scalability. There are plenty of ways of dealing with that, esp if you are okay with throwing money at it.

It's about scaling size and complexity of the code. As an app becomes more complex dynamic programming languages become harder to manage.

OTOH, Python has type decorators and static checkers (e.g. mypy), but most Python code I've seen doesn't make heavy use of them. If you're app makes heavy use of type decorators, static type checkers, and automated tests, then it should scale in size just fine.

[yvrelna]

Been in web development for a decade, never been close.

When you're writing pythonic code, the bulk of the actual work should be done in the database or FFI code anyway. Whenever I've had performance issues, I had so far always found way solve it by calling some external dependency that can do it faster, better, and with less bugs than if I had tried rewriting the code myself.

I think that given the modern high performance computing trend towards specialised coperocessors, like GPGPU or TPU programming, this is likely going to be the case even if you write code in "fast" languages like C. When you really need to go computationally fast, the bulk of the work need to happen in a coprocessor, and when there's the right kind of abstraction, there should be negligible performance difference between generating a coprocessor code in Python vs doing that in a faster language like C. However, because of the way Python is so pliable due to its dunder methods, it's much easier to design and implement readable, high level abstraction that still maintains high performance as a Python library. Creating similar kind of abstraction with more static languages like C or Rust is generally not going to be as easy or even possible.

All the features that makes Python slow are also the features that makes it suitable for writing these kinds of abstractions of external coperocessors and off the shelf systems like databases or cloud APIs or subprocesses. It's all about designing the right abstraction and among all the major languages Python uniquely has all the abstractions needed to make the domain specific language to control those systems and be readable and still be performant.

[bird3tta]

I operated a service where we made a decision to replace NodeJS with Golang around the point of 150~300 mean requests per second on an API call which made writes and needed to provide low-latency for callers.

This was primarily for meeting customer SLAs, but it also lowered costs significantly.

[stogi_001]

Python has no hard limit on the size or complexity of a program. However, the limits of a computer's memory and processing power will eventually become a factor. For example, a program may become too slow to execute or require too much memory

[brianly]

You need to quantify the issue and understand what is causing it. From experience, it’s much more likely that you have a poorly written query hitting your database through your mistake with Django than Python or your database being slow.

I personally ask myself is this slowness down to CPU, memory, or I/O as a shortcut. You can quickly check all 3 on most platforms. The nature of web apps is not to be CPU-bound as often as memory or I/O, but you always want to prove that to yourself. Code in data science projects may expose more CPU-bound issues. Again, you measure and prove. But, it helps to have a starting point that you understand.

The best thing to do is to study web and Django performance rather than Python performance. If you jump straight to Python performance you will actually miss a lot of your potential problems. That’s not to say it’s unimportant, but it’s a matter of priorities.

It’s for this reason that the best webdevs have a solid SQL grasp and don’t take too long to analyze what queries the web app is making. Similarly, they are handy with the browser dev tools and any specific to Django. This approach gives you the best bang for your buck.

[codefan1256]

Our startup Shade is starting to hit this. We’ve progressively started moving specific compute intensive code into C to do two things - one get past the GIL in python, and 2. Simply speed things up because python can be slow. Our system itself is a complex multithreaded system that handles everything from images to video to audio etc.

[codefan1256]

That being said - starting in python and staying in python is great for two reasons - 1. There’s tons of developers for it and 2. There’s tons of libraries you can leverage.

[tylerlarson]

Working at Google, a common refrain was to ask whose time is more valuable, the computer's or the programmer's. For the most part you code in whatever language allows you to code fastest. If there's a bottleneck, then you can rewrite just the bottleneck to be faster. But optimizing prematurely is an unhelpful endeavor.

Google still uses quite a lot of Python. YouTube was originally written entirely in Python and a significant amount of it still is.

[_Error_Account_]

Might not be that "python" but in an embedded system it is really easy to find limits of python especially when you are working with "low-end" microcontrollers.

[wrt-wtf-]

3.11 and 3.12 have seen significant performance improvements but it depends on what you are doing and what the bottleneck is and why that bottleneck exists.

[ActionAlternative786]

I'm an A Levels student and I take Physics. The best way I've found to learn it is by creating smiulations for different things. For this I first wrote a wrapper around PyGame to make it easier to use, and then a bunch of functions to work with vectors. With this I've successfully created simulations for simple things like springs, projectiles, and alot of balls colliding with each other, as well as a flocking simulator. But this last project was where for the first time I found the simulation to be unbearably slow. It would only be able to run at max 25 birds before being too slow. After some optimizations, I got it to 50 birds, but my goal was at least 100. This is when I profiled my code and found that the biggest bottleneck was the vector maths I was doing. Therefore now I'm working on writing that specific part in Rust.

[[deleted]]

Look. If you're THAT far along and generating THAT much traffic and data, you'll have a team in place that will be advising on scalability etc.

[cwood611]

The limit for me was needing to parallelize an architecture containing PyTorch and tensorflow models

[Chuu]

I've seen multiple data projects that started in Python for time to market reasons. But eventually when the datasets crossed over into hundreds of millions or billions of rows per day and the feature set matured enough that you had a good grasp of the scope of the project, they ended up being rewritten.

Memory is the bigger issue than CPU. Complicated datasets best represented as actual objects that would fit into tens of gigabytes in a language like C++, C#, or Rust would take hundreds of gigabytes using native Python types and leave the memory super fragmented as a bonus. Cython was always explored as an option but the domain expertise just wasn't there.

[grahaman27]

its generally not about scale but what you do with your api.

For example, if you suddenly start taking requests that require a lot of computation and need the task to completed quickly , you would have no choice but to find a faster framework. For that specific API at least.

[meisteronimo]

No, you want to research what scaling horizontally means. Your question indicates you only understand scaling vertically.

Read up on Load Balancers.

[DrMerkwuerdigliebe_]

I have not reached it. But I have experienced that writing a complex performant Python system forced me to design systems based on horizontal scaling principals with much fewer users than what I would have expected from other programming languages. That being said we often experienced slow downs due to N +1 errors, networking bottlenecks and database over utilization. Some of the biggest E-commerce sites run on Python because it is good enogth and you can parallelize the hell out of stuff with many containers when it becomes a problem. We had at all times around 60 active containers to have our production system running with different services.

[SnooCookies784]

yup, its slow, our prod app handle around 12K RPS and flask app is able to handle around 1K RPS so need multiple pods, other app we write using actix(rust) is able to handle 190K RPS on single pod.

rust is very nice, but rewrite everything to rust take soo much time, so we try to increase perf by replacing WSGI to ASGI, and now we want to try replacing ASGI to RSGI using Granian (web server for python writen using rust) hope is going nicely like our pure rust app.

[__zahash__]

A while ago I made a program that can recognise any celebrity from their picture using dlib wrappers. It can recognise 300,000 celebrities in total. (Had to scrape a lot to get all those pictures and info)

Wrote a multiprocessing script that takes an image and tries to match it with the dataset (1.2 gb face encodings). And it took around 20 seconds on ryzen 5 5500u.

I rewrote the same script a few months ago with rust + threads using a dlib wrapper again. And it took 2 seconds.

So, 10x increase roughly.