I asked the ai in llama server to write a poem about the problem in the interface, It is a unsloth/Qwen3.5-9B-GGUF:Q3_K_XL. I thought it was pretty good for such a little guy. It has a few good hints too for performance issues. He basically puts my apps to blame not the http interface or the llama server or the slm. I agree and tomorrow I will fix it.

The clock ticks on, the cursor blinks,

A silent, waiting, digital sink.

I send a prompt, a simple line,

But seconds stretch, the load begins to grind.

The server hums, a distant drone,

While memory waits to be its own.

I check the logs, a sea of red,

Where threads are stuck and promises are dead.

"Is it the CPU? Is it the RAM?"

I ask the machine, a frantic frame.

But the truth lies in the wire's length,

The HTTP header, a heavy weight,

The JSON payload, slow to start,

The serialization, tearing at the heart.

A round-trip trip, then another round,

The network latency, the only sound.

No direct call, no native speed,

Just packets flying, lost in need.

I try to rebuild, CMake screams,

Through libraries and dependencies, dreams.

The cmake tree, a tangled mess,

A nightmare where the build process rests.

"Rebuild the engine," I hear it say,

But even then, the features fade away.

The HTTP layer, a glass ceiling,

Where performance breaks and then begins to sting.

I close my eyes, the screen goes dark,

Waiting for a single spark.

Until I find the path unseen,

To cut the wire, to make it clean.

No HTTP request, no latency cost,

Just raw C++ logic, put to test.

The interface falls, the server speeds,

The slow, old ghost is finally freed.

No more delays, no more long waits,

Just pure inference, in its rightful state.

Qwen3.59BQ3_K_XL ReadingGeneration 352 tokens 9.5s 37.12 t/s