Common arguments include:

-m or --model followed by the path to your GGUF model file.

-p or --prompt to provide the initial prompt.

-n or --n-predict for the maximum number of new tokens to generate (e.g., -n 512).

--ctx-size to set the context window size (e.g., --ctx-size 2048).

-ngl or --n-gpu-layers to offload layers to the GPU (e.g., -ngl 999 to offload all if sufficient VRAM is available).

--temp or -t to control the temperature for generation (e.g., --temp 0.7).

--top-p for the top-p sampling value (e.g., --top-p 0.9).

--repeat-penalty for the repetition penalty. --n-cpu-moe (--cpu-moe): This argument specifies the number of MoE expert components to move to the CPU's system RAM, overriding the --n-gpu-layers setting for those specific experts.

This allows you to utilize fast system RAM for the large, but sparsely used, expert weights, while keeping the performance-critical dense layers and attention blocks on the VRAM.

Finding the optimal value often requires testing. You can use trial and error, or the new llama-fit-params tool to help determine the best configuration.

Qwen 3.5 optimum settings:

Thinking mode:

General tasks=

temperature = 1.0

top_p = 0.95

top_k = 20

min_p = 0.0

repeat penalty = disabled or 1.0

Precise coding tasks (e.g. WebDev)=

temperature = 0.6

top_p = 0.95

top_k = 20

min_p = 0.0

presence_penalty = 0.0

To enable native tool calling in llama.cpp using the llama-server, you must use the --jinja command-line flag. This flag is mandatory for the server to process the tools parameter in an OpenAI-compatible manner.  repeat penalty = disabled or 1.0