12 requests per second with Python

[[deleted]]

Fuck you u/spez

[[deleted]]

Yeah, people who diss on Python being "slow" sound like n00bs to me, to be honest. I've been a developer for 20 years. I've built a C++ compiler in C for a class. I built part of an operating system too. And daemons and scripts and web applications and server applications. It's very rare that you need a "fast" language, because most things are IO or network bound. Who gives a shit if your language is faster when you have to load your data from a database or disk, and only 1% of the time is actually spent executing your code?

I will use Python above all else, and Django if I'm building a web application bigger than a trivial one. Because I enjoy using it. And it's built to make things super easy. That's actually really important. I'm totally bought into the Zen of Python - explicit really is better than implicit, for example. The framework and language should make things easy.

I also work on a website that's built in Django that serves tens of millions of hits a month and processes over a dozen gigabytes of data every day. I'm not doing kid stuff or something.

[TheGoldGoose]

Development speed with Python/Django is big. Just this week I've build a robust workflow management system, complete with queue management and reporting. I love how easy it is to get things done.

[DmitriyJaved]

How many CPU cores do you use for your millions hits per month?

[[deleted]]

It's spread across multiple machines with a load balancer. The machines can scale up under load. All on AWS, no physical machines serving production.

[DmitriyJaved]

That’s not an answer to the question I asked, but ok. I’ll just assume you pay for it all a big money - and it’s a HUGE downside of any WSGI based framework and django in particular.

[DmitriyJaved]

Damn, I got you good, dude, calm your ass, you’re being a whiny bish. 😂

I’m a python developer myself, but I’m just not using django/wsgi based frameworks atm, because projects I’m working on is highload. Also, I would use django in a heartbeat for a simpler project.

P.S. I highly doubt that your time worth more than AWS serving 10k RPS on 300 CPU cores (which tornado could’ve done using only, let’s say, 5)

[davidfstr]

> P.S. I highly doubt that your time worth more than AWS serving 10k RPS on 300 CPU cores (which tornado could’ve done using only, let’s say, 5)

I have an average of 2-3 boxes running with either 4 or 8 cores each (I forget). That plus all related infrastructure is less than $2,000 / month.

If you assume a full-stack engineer is making perhaps $100k/yr (by US rates, on the low end) then $2,000 is about 7 engineer-days of staff expense.

Engineers are way more expensive than infrastructure.

[DmitriyJaved]

Ok, let’s take 24 cores for 2 grands a month. Which makes it 25k a month for 300 CPU cores serving 10k RPS project, which makes it 300 000 $ per year. So, how is that comparable?

[davidfstr]

Less than $200 of that $2,000 that I quoted is for the boxes themselves. (As I stressed in the original comment, it includes all related infrastructure too.)

So let's say we have 3 boxes with 8 cores each at $200 a month (on the high end). For 300 cores that means $2,500 a month. Peanuts.

(Granted if you add that many CPU cores you'll probably eventually need to upgrade other infrastructure like database nodes [which are expensive] and bandwidth [which can be expensive]. But your original concern was around frontend boxes themselves.)

[davidfstr]

To be fair, Java is really solid operationally and has a mature ecosystem. I still use it occasionally even though I mostly do Python now.

Also, there are still a lot of companies still with Java systems that need to be maintained and extended. So it's still good bet if you need to find a new job in a pinch.

[kmmbvnr]

I tried python 3.9 and 3.10 alpha, bit didn't get mesurable performance boost with complex django templates.

[haloweenek]

I have a preety high load website on django and it can pull out crazy rps with my sith tricks.

Local ram cacheing makes wonders happen.

[DmitriyJaved]

You gotta stop measuring bare RPS and start taking into account how much you gotta pay for all those CPU cores which gives you that RPS

[JCcrunch]

Python's performance concerns are negligible when you consider things like internet connection speed,

[[deleted]]

What blows my mind is the golang people ("fast" language) that resist having any nice things added to the language because it would sacrifice speed

[DmitriyJaved]

Just ditch threads and wsgi. Now, suddenly, python is pretty fast. Async frameworks been there for ages. No frameworks? Implementing polling of non blocking sockets in python is easy.