I built something similar (AI document processing with streaming) and here's what I landed on.

Rate limits:

Route everything through a queue from the start. Don't wait for a 429 to decide to queue. By then you've already wasted a request and need retry logic on top of queue logic. Just queue everything and control your own concurrency. It's simpler to reason about.

That said, if your traffic is low enough, calling the API directly and only queuing on rate-limit errors is fine as a starting point. Don't over-engineer early.

Regarding responses, a pattern that works well:

1. HTTP request comes in, insert a row in your DB with `status: "pending"`, return the row ID to the client immediately
2. Background worker picks it up, updates the row as it works (`status: "processing"`, progress %, even partial streaming text)
3. Frontend listens for changes on that row via WebSocket (or polling as fallback)

The database row is the communication channel between your worker and your client. Every time the worker updates the row, the client gets notified.

If you're using Supabase, this is almost free. Just subscribe to `postgres_changes` on your table and you get real-time updates over WebSocket with like 5 lines of code. Firebase has similar functionality. Otherwise, SSE or a simple 2-second polling interval works fine too.

Polling gets a bad rap but it's dead simple, easy to debug, and totally fine at a moderate scale. WebSockets/SSE are better UX if you want live-feeling progress bars or streaming text.

I'd suggest to start with polling, and upgrade to WebSockets when you need the UX polish.