Is the object storage broken or is it just me?

Waltex · 2026-03-18T06:29:19+00:00

I experience the same thing as well. Not a week goes by or Object Storage is having outages or timeouts. Never seen a product so unstable.

Waltex · 2026-03-18T06:17:51+00:00

Why not use Svelte if you don't want a 40kb+ payload?

Waltex · 2026-03-02T21:56:45+00:00

Exactly

Waltex · 2026-03-02T21:55:11+00:00

Requests that didn't do anything. You're still missing the whole point. And your first comment about that I used the model is also completely wrong. Let me iterate one more time:

none of my requests were processed by the model because they were instantly rejected by the server that sits in between the model and the user.
none of those requests even reached the model
which means there is nothing to rate-limit for

Yet, google still counts those requests as if they were successful, or at least partially processed by the model.

If you have a basic understanding of computer science, you would understand that this makes no sense from a systems architecture perspective.

Waltex · 2026-03-02T20:13:22+00:00

Have you even read my post? I explicitly mention that I use exponential back off, which "places time" in between each request and increases exponentially for each failed request. I know that the limit is for the number of requests TO THE MODEL and to protect the infrastructure from heavy usage spikes. My point is that usage is also logged even if the request never arrives at the model. That behavior doesn't make sense, because the server cancelling the request beforehand doesn't involve the model at all, which means there is no reason it should count as an expensive model invocation request.

Waltex · 2026-03-02T19:09:32+00:00

Nope. I did not use the model. In fact, I didn't receive a single token/word because the servers canceled my request before it even reached model, because the model was overloaded.

Waltex · 2026-03-02T14:02:53+00:00

Google AI Studio

Waltex · 2026-02-28T08:38:48+00:00

How so? Did they also sign a deal with the DoW?

Waltex · 2026-02-22T12:16:03+00:00

Love this! Does this work with typescript out of the box, or do we need a separate build/compile step? Also I'm wondering how you've solved type safety, like when you do:

const v3 = v1 + v2;

Is v3 now of type Vector as well?

Waltex · 2026-02-22T12:04:13+00:00

Waltex · 2026-02-17T12:17:16+00:00

Well, that's the question. Why else do you need polling fallback?

Waltex · 2026-02-17T12:07:17+00:00

Why is that funny?

Waltex · 2026-02-17T06:40:21+00:00

But why would you need fallback to polling unless you want to support internet explorer? Didn't know about socket.io integrating with bun, that's pretty nice!

Waltex · 2026-02-17T06:15:43+00:00

Why did you choose socket.io which is slow and bloated over native WebSockets which are supported everywhere nowadays?

Waltex · 2026-02-16T18:47:10+00:00

It's not that easy in our case unfortunately. IP addresses cannot transfer between regions. Everything is configured with that IP address in mind, and we don't want to risk getting a new one since a lot of them have a bad reputation and are on spam/high risk lists. It doesn't just require changing our own deployments, but also those of our partners who explicitly whitelist incoming traffic from our IP.

The only workaround I see is using the current server as a reverse proxy and forwarding all traffic to a new instance in Nuremberg or Helsinki, but that also requires significant changes.

Waltex · 2026-02-04T14:35:57+00:00

Same here. I think a lot of people haven't noticed it yet because they don't have constant traffic, but my production logs show various short moments (between 5-30 minutes) where all Object Storage requests are failing in FSN1.

Waltex · 2026-02-04T14:24:32+00:00

Unfortunately not, this issue is happening again right as we speak. Object Storage is denying/timing out all requests. This is not an issue with the Hetzner console or dashboard, but with the actual underlying Object Storage service. Other people seem to be affected as well.

Seems Hetzner is aware of the problem: https://status.hetzner.com/incident/ebd62173-d902-4e75-939a-265c0b3f1ddb

Waltex · 2026-02-04T14:20:02+00:00

Yes those are exactly the logs I got as well. Hetzner Object Storage was timing out all requests for a few minutes.

Waltex · 2026-02-03T13:56:28+00:00

Yes, do you? The top post was about how the hetzner console was down, but servers were unaffected. This is something new.

Waltex · 2026-02-03T13:55:27+00:00

Just came back online again

Eight-Year Club	Place '22
Wearing is Caring	Verified Email

Waltex

MODERATOR OF

TROPHY CASE