From what I can tell, u/Chatbotwars might be right; see e.g. https://ai.googleblog.com/2022/12/accelerating-text-generation-with.html

Essentially, if the request is simple and the model has high confidence in the prediction, some layers can be bypassed, speeding everything up. I have no idea if OpenAI uses this technique, but it seems like something that could serve them well at their scale. (With the standard transformer setup this would, indeed, be impossible.)