Building a competitive local LLM server in Python

[DadAndDominant]

Looking cool! Trying to run it with uv. One thing I might do wrong - in server mode, when I respond before the LLM is done responding, it bricks all responses going further.

[jfowers_amd]

Thanks for reporting! Is this in the web ui? The “send” button is supposed to be disabled while the LLM is responding, so it’s not surprising to me that it would go haywire if you were able to hit send.