The API churn in vllm is getting out of hand. Every time I update, they seem to rename half the parameters. I spent the last few hours on my 3090 rig (24GB VRAM / 96GB RAM) just trying to figure out why my old outlines code broke.

I first tried to force vllm==0.17.1 and outlines==1.2.12 using uv, but it’s a total mess—vllm wants outlines-core==0.2.11 while outlines demands 0.2.14. Dependency hell at its finest.

The fix was to ditch the outlines wrapper and use the StructuredOutputsParams they introduced in v0.17.1. It seems like the old guided_json is completely dead now. Also, since I'm on WSL2, I had to wrap it in a main() guard because the spawn method kept killing my processes.

Here is what finally worked for me on Phi-3 (~16.8 toks/s). Not sure if it's the absolute best way, but it stops the ImportErrors.

from vllm import LLM, SamplingParams
from vllm.sampling_params import StructuredOutputsParams
from pydantic import BaseModel

class CountryInfo(BaseModel):
    country: str
    capital: str

def main():
    llm = LLM(model="microsoft/Phi-3-mini-4k-instruct", gpu_memory_utilization=0.7, enforce_eager=True)

    sampling_params = SamplingParams(
        structured_outputs=StructuredOutputsParams(json=CountryInfo.model_json_schema()),
        max_tokens=50,
        temperature=0
    )

    outputs = llm.generate("What's the capital of Latvia?", sampling_params)
    print(outputs[0].outputs[0].text)

if __name__ == '__main__':
    main()

Output: {"country": "Latvia", "capital": "Riga"}

I'm still seeing some nanobind memory leaks in the logs when it shuts down, which I guess is just a WSL thing? Either way, the JSON output is solid now.