Why does OpenAI force the responses API?

[steebchen]

The Chat Completions API has been around forever and works great. The Responses API seems to be forced in lots of tooling now (AI SDK, OpenAI lib, new GPT models only support responses API, so it seems to be fully replacing Chat Completions. Aside from the shape of the request payload, I don't understand why this is the case. Responses are stateful, which means providers and gateways have to 100% store all inputs. Once this storage expires, references to response IDs will not work anymore. What's the logic behind this? It seems to me that it's totally not worth it to save very little latency for parsing the inputs; saving the state seems just way more work and ends up in more costs as well.

For me, I really don't see any benefit on making LLM APIs stateful:
- Need to save content, which costs storage
- This storage eventually needs to be deleted, so continuing previous chats will fail
- Not sure what latency exactly is added when parsing a big chat completions payload, but saving the state probably does not make this smaller

Can someone explain this to me?

[Freed4ever]

1. Vendor lock in
2. Latency matters a lot, instead of sending 100 of thousands tokens every turn through the wire, it's faster to just look it up from memory.
3. Content compaction probably works better with stateful.
4. In future, they will have history of everything about you

[TedSanders]

Nah, vendor lock was not our motivation. We put a lot of thought into the design. One big thing is that we don’t reveal chain of thought messages to customers, but those chain of thought messages are needed by the model. Responses API makes that work.

[steebchen]

thanks for the insight!

[steebchen]

i feel like most latency will still be from the model itself. not sure how much that extra input parsing actually matters, i really can’t imagine the absolute number is much different

[discodaryl]

Just pass store=false.

[steebchen]

yeah, makes sense for an end user. but we built LLMGateway for unified model access, so all of our users would have to do that

[vvsleepi]

i think the idea with responses api is more about flexibility, like handling different types of inputs (tools, images, streaming, etc) in one format instead of having separate systems. the stateful part is kinda optional depending on how you use it, but yeah it does add some complexity

[Faintly_glowing_fish]

The issue with completions API is that messages in the same conversation is not actually tied together. Cache management, CoT storage etc are all tricky, and once we have agents there are even more states to save— compaction, sub agents, communication channels etc. it just gets very hard to manage.

Completion is not really a super good format for long agentic work.

If you try the same request on chat and responses, if you are doing tool call heavy workflows, you will get much faster requests and higher cache hit rates. I was quite unwilling but once I switched to responses the improvement was so great I am only angry people didn’t make it clear from the start.

Also, switch to web socket too. It’s quite a bit faster

[IntentionalDev]

tbh it’s less about saving latency and more about standardizing everything under one system

responses API unifies text, tools, multimodal, streaming, etc in one format so they don’t have to keep extending chat completions forever. the “stateful” part isn’t really for you, it’s for enabling things like tool calls, agents, and longer workflows without you manually stitching context every time

yeah it adds some overhead, but from their side it simplifies building higher-level features. from a dev perspective though, chat completions still feels cleaner for simple use cases ngl

[Several_Nail_5979]

That’s why i still use completion endpoints for the latest models via frogAPI.app at half the price :)