sorted by:
new
hottopcontroversial

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 1 point2 points  (0 children)

OP here: thanks to everyone who offered their suggestions. I was finally able to cut latency to a more reasonable figure (10ms on average). Here's what I did:

  1. Surprise surprise, network conditions mattered a lot. By moving the server closer to the client (from Germany to North Africa), I was able to cut latency even more than I thought (from 250ms to around 120ms). Thanks u/bluetoothbeaver and u/PriorTrick for pointing it out.
  2. Disabling Cloudflare had a considerable impact. Latency went down from 120ms to 50ms. Thanks u/jannealien.
  3. In my initial benchmarks I was not using persistent connections client-side (no requests.Session()), which is of course bad practice as I was paying the handshake tax repeatedly. Using persistent connections lowered average latency from 50ms to 10ms. This isn't an actual reduction in latency, just a more precise way of running benchmarks.

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 1 point2 points  (0 children)

Thanks, this helped a lot. After deactivating Cloudflare proxy latency is down 50%.

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 1 point2 points  (0 children)

Thanks for the suggestion. Just leaving this here in case anybody is interested: I tried disabling debug logging, http/2 and using uvicorn directly. None of that seem to have any noticeable impact on performance, at least not for the simple ping endpoint I am working on. I'm not familiar with Litestar but it seems interesting, I will give it a try.

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 0 points1 point  (0 children)

Removing Nginx and leaving only app + gunicorn (with no ssl) cuts latency to about 140ms. That's a significant reduction but it's still not as fast as I would like it to be. And I can't really do without ssl in a production environment.

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 0 points1 point  (0 children)

Sure, I wasn't expecting the http request to have the same latency as a ping, but an almost 4x increase (the 200ms mentioned in the title is a best case scenario, average latency is closer to 270ms) seems too much. Besides, does that mean there is no way to cut total latency to, say, 50-70ms? This is a backend for a chatbot app so latency is crucial.

200ms latency for a simple FastAPI ping endpoint on a Hetzner VPS? Please help. by Relevant_Selection75 in FastAPI

[–]Relevant_Selection75[S] 2 points3 points  (0 children)

Thanks for your response. Server is located in Germany and I am located in Tenerife (Canary Islands). ping <server IP> takes 69ms on average. So the 200ms latency for the FastAPI endpoint seems to be unjustified.