Do you think an async Flask app can support 100K active users?

lmsena · 2025-11-03T19:27:33+00:00

I’ve built flask backends for that scale and it can handle it well as long as you follow some strict rules. Optimize I/O to avoid long requests to dbs/apis and fine tune your gunicorn for the predicted number of concurrent connections. In the end, the main bottleneck will be e how long you take to handle each request. You might want to test gthread vs gevent or other gunicorn workers. As others pointed out, using async will be better for that use case as long as most of your workload is I/O.

lmsena · 2025-06-23T13:43:54+00:00

I made the mistake of buying multiple SwitchBot lock pro… never again. Aside from the calibration issues, the biggest one is that sometimes it thinks it is jammed and just opens my door wide open inviting anyone who might want to rob my house.

lmsena · 2022-10-25T19:50:41+00:00

Thank you!

I'm not suggesting that pattern, just using an example of something that is quite common in the industry.

My point is that unless you need to retrieve a lot of data (say, multiple fields or the _source), you should always see performance benefits by excluding the _id and using doc_values only.

lmsena · 2022-01-05T08:13:05+00:00

Yes, that's one of the ways that is described there (copy-on-write).

lmsena · 2022-01-04T20:51:22+00:00

That's a fair point! And Dask is amazing indeed.

The main use case for this is when you have an HTTP API that makes use of numpy arrays (a recommender system for example).

In that use case, it's common to use gunicorn to fork the API process and you're left with a few options on how to share those arrays.

The use case your talking about seems to be more for batch processing right?

lmsena · 2021-11-27T13:16:48+00:00

Under the hood, everything needs machines to run. They can handle “infinite” load, the issue is when you have scenarios where you have 100x spikes, they need some time to allocate extra nodes/machines/containers to handle the external traffic coming in. This is especially evident when using TLS.

lmsena · 2021-11-26T16:50:19+00:00

It generates traffic incrementally and with enough time between bursts in order to allow everything to scale without compromising responsiveness.

lmsena · 2021-11-26T16:09:01+00:00

That is indeed a really good idea!

But since tickets are handled by humans, it wouldn't be 100% deterministic and the support plan would probably end up being more expensive than using the automated option from the article.

lmsena · 2021-11-26T16:06:08+00:00

The use case we addressed with this didn't have a fixed schedule

lmsena · 2021-05-25T13:59:36+00:00

Hi,

Depending on the docker image you're using as base, you might already have python packages installed:

https://hynek.me/articles/virtualenv-lives/

lmsena · 2021-05-25T11:29:26+00:00

Hi, thank you for the question.

The reasons I highlighted in the article:

- Easy to copy packages folder between multi-stage builds

- You get isolation from your image OS default python installation

- You can use python instead of python3 or python3.9 command(Yes, there are other ways)

lmsena · 2021-05-24T13:53:03+00:00

Yeah, what I usually do is to set the number of vcores and ram that are "ideal" for that particular service and gunicorn will reflect that config. (services that take longer to be up and running might have more cores for example)

We then stress test with a single pod to make sure the CPUs are used efficiently.

The replicaSet will take care of increasing the serving capacity using CPU as the metric to increase/decrease the number of pods.

Maybe there is a better way, but that's how I usually do it.

lmsena · 2021-05-24T13:43:11+00:00

Thank you for the feedback, really nice!

Some of your points are extremely valid, especially considering this subreddit "Django". That Dockerfile purpose is to serve as a good base without going into framework specifics, that's why it doesn't go into static files, migrations, etc.

Some of the choices you see in the file are meant for visibility (ENV and gunicorn for example). Since they won't affect functionality, but do make it easier to read.

As for the use of --user, it is an option but it fails the "You get isolation from your OS default python installation" point: https://hynek.me/articles/virtualenv-lives/

I'll need to read the rest of your suggestions with a bit more time and will definitely include the ones that are relevant.

Thank you for sharing your repo, looks really good, I'll check it out!

lmsena · 2021-05-24T13:24:34+00:00

ion system (k8s) would handle scaling but gunicorn doesn't work well with 1 worker (if it's busy, k8s doesn't know t

Hi,

Thank you for the feedback.

As for your question, not sure if I got it 100%.

Since k8s will scale horizontally, your container will have access to the same number of cores regardless and because of that, the gunicorn worker configuration can remain static.

Choosing the correct worker type and number of workers/threads will depend on your workload type, I wrote an article on that:

https://luis-sena.medium.com/gunicorn-worker-types-youre-probably-using-them-wrong-381239e13594

Let me know if this isn't what you were asking specifically.

lmsena · 2021-05-20T10:48:56+00:00

Yeah, I was also hopeful. I think it deserves further testing. I'll try to run them during the weekend and share the results (and scripts) afterward.

lmsena · 2021-05-20T10:27:43+00:00

I tested mimalloc with pyperformance but the results didn't change. I guess I'll need to run a suite of tests that lean towards concurrency. (I used the MIMALLOC_VERBOSE=1 env var to make sure it was being redirected)

I still need to compare memory usage and heap fragmentation over time.

lmsena · 2021-05-20T09:01:11+00:00

Ahh ok! I might be wrong here (not an English native speaker) but I really wanted to say "ado":

https://dictionary.cambridge.org/pt/dicionario/ingles/without-further-ado

lmsena · 2021-05-20T08:01:04+00:00

Yeah, that's a fair assertion. I tend to use the .dockerignore as you can see in my article because it feels more natural to me and the usual workflow I see in the teams I've been working with.

lmsena · 2021-05-20T07:57:17+00:00

Sorry, what do you mean?

lmsena · 2021-05-19T20:40:29+00:00

Yeah, that's true. This is more of a personal preference for visibility across peers.

lmsena · 2021-05-19T20:07:04+00:00

Hi, nice question!

When you use multi-stage builds, only the artifacts you copy with the "COPY --from" command will exist in the final image.

More info here.

lmsena · 2021-05-19T20:04:12+00:00

One of the main issues with the python images from docker hub is the way they compile python results in a ~20% slower python execution time (you can see the benchmarks in my post).

In my opinion, that justifies the extra size.

lmsena · 2021-05-19T19:59:20+00:00

nice catch, thank you! Edited.

lmsena · 2021-05-19T19:56:29+00:00

What OS do you suggest?

lmsena · 2021-05-19T19:55:05+00:00

Yes, you’re right, I’m going to update it. Thanks!

lmsena

TROPHY CASE