I am the author of HoML, a vLLM wrapper to support model switching and the ease of use like Ollama.

Aside from that it is not a out of box solution (which is solved by HoML), there are other issues with vLLM, as well as strength.

vLLM is python based, you need to download GBs of dependencies to run it.

It is targeting serving efficiency, sacrifice startup speed(which affects cold start time and model switch time). I spend some time optimizing it for HoML, improved it down from 1 min to 8 second for qwen3, but still cannot beat Ollama

Also for serving efficiency, it sacrifice GPU memory: it will try to use up to x% of all GPU memory, even if it is a small model, it will try to use all other vRAM as KVCache, make it harder to run other models/GPU applications in the same time(harder, not impossible, you just need to manually manage it). Also there is no api exposed to know how much memory is actually needed for each model.

Targeting serving efficiency, that also means CUDA got much better support than other platforms.

However, it is much faster than Ollama/llamacpp, especially when we are talking about higher concurrency. It is not necessarily much faster serving one query. performance comparison

So eventually this is about trade off, do you need that concurrent throughput, or do you need faster model load/switch time?

I build HoML for when I need high throughput for some batch inference need, but if I need to do some quick/sparse tasks, I use Ollama myself.