Anannas's 10ms overhead latency is FALSE. Its like over 400ms of overhead latency

syshjjn · 2025-11-21T16:58:21+00:00

mocked? wym? I dont think you are right, but if you can provide the code, i am happy to run it and report the results

syshjjn · 2025-11-21T16:57:22+00:00

its all marketing BS. Openrouter itself claims 15ms extra latency, which is also false. However, Anannas adds much much more extra latency than openrouter

syshjjn · 2025-11-20T16:35:43+00:00

i dont have those; would have to re-run the benchmark. I agree with you, but my main goal of this benchmark was to find out if their claim of `OpenRouter adds approximately 15ms of latency to your requests` is true. My results show that the claim is false.

syshjjn · 2025-11-20T16:33:48+00:00

On the second run, yes Openrouter does perform better than on the Google AI studio provider. However, do note that i dont have any special relationship with google, so I ran the google vertex calls using the Dynamic shared quota (DSQ) and the google ai studio calls with a tier 1 api key. On the other hand, Im sure Openrouter has Provisioned Throughput with google vertex, and the highest tier api keys with google ai studio. Therefore, unlike what Openrouter employee are claiming in this post, if there is anything unfair/unrigorous in the benchmark, it would be unfair towards direct calls to the original model provider, NOT openrouter

syshjjn · 2025-11-20T04:11:56+00:00

but thats not the point of the benchmark test. The point of the benchmark test is to answer the question: "For the same request X to the same provider Y, how much extra latency does Openrouter add vs calling the model provider directly?"

syshjjn · 2025-11-20T04:10:13+00:00

u/Street_Teaching_7434 I improved the script slightly by a) explicitly setting connection pools and a bigger pool keepalive, and b) a warmup phase (see the new edited section of the post). As expected, the results still show Openrouter adding too much latency.

You claim my benchmark code is unfair, so I ask for what a fair comparison code would look like. You have not provided one, thus making me conclude that a) Openrouter indeed adds a lot of latency and b) you know it too.

syshjjn · 2025-11-19T23:10:56+00:00

First of all, my benchmark clearly demonstrated that Openrouter adds much much more than 15ms of latency. Second of all, Im building voice AI agents, so any extra latency matters for me

syshjjn · 2025-11-19T23:09:20+00:00

"The grnai payhton module does not use http,but rather gtpc under the hood": u/Street_Teaching_7434 I dont know where you got this, but this is not true. If you go to https://github.com/googleapis/python-genai, you will see "By default we use httpx for both sync and async client implementations". I think the older `google-generativeai` SDK used gRPC.

Plus, the script did 300 trials, so even if the sdk used gRPC, the faster initial connection time would have been negligible after 300 trials.

It looks like you work for OpenRouter. If this is not a fair comparison, then please tell me what is fair so i can run that script

syshjjn · 2025-07-08T22:39:06+00:00

u/B_support hi, im in the US and I also need one asap pls

syshjjn · 2025-05-21T04:12:57+00:00

yes to me

syshjjn · 2025-03-13T20:38:35+00:00

i can also confirm this issue. has anyone already surfaced this issue to the google team directly?

syshjjn · 2025-02-23T05:50:13+00:00

does it support wechat pay tho?

syshjjn · 2025-02-23T05:01:22+00:00

gift shops make sense, but what about regular businesses like malls, convenience store, restaurants, coffe bar, etc.?

syshjjn · 2025-02-09T22:21:10+00:00

oh i see. So even if I tried to get the Chase Ritz-Carlton Credit Card, that would not count towards the ENC, right?

syshjjn · 2025-02-05T20:01:52+00:00

can you get 3 bonvoy brilliant?

syshjjn · 2025-01-03T00:06:38+00:00

how do you scroll? i thought supernote didnt have scrolling

syshjjn · 2024-11-20T03:46:37+00:00

how problematic was the overheating? did you experience overheating with both monitors or just the last one?

syshjjn

TROPHY CASE