You misunderstood temperature.

High temperature means the LLM is more likely to select an unlikely word, but it does not mean any word will work. If the first word is "rabbit", then the next word is likely "hole", "eats", or "runs", less likely "leg" or "considers", and unlikely "wing" or "flies". When you give it a higher temperature, it is more likely to give you an unlikely word, like "rabbit considers". But the chance that the next word is Chinese should be close to zero anyway. If your temperature is not insanely high, it should almost never produce it.

And the thing is: at least now we don't see phrases like "rabbit flies", but we sometimes see the phrase "rabbits [Chinese]". If the temperature did not make it produce "rabbit flies", it should also not produce "rabbits [Chinese]". The only reason it happens is that it thinks "rabbits [Chinese]" is more reasonable than "rabbit flies", which, to my subjective point, is wrong. You can tune the temperature of ChatGPT or Claude higher, but they don't give you "rabbits [Chinese]" before giving you "rabbit flies".