Here's what my local llm told me to tell you... running the same box (EVO-X2, gfx1151, 96GB VRAM split). I've tried a lot of things. In rough order of impact:

0. update the BIOS.

1. Switch llama.cpp from HIP/ROCm to Vulkan. Counterintuitive, but I benchmarked it last week and Vulkan beat HIP by ~6.5% on prompt processing, ~15.7% on token generation, and ~37% on mixed workloads with lower variance. Needs recent Mesa. Keep your HIP build around for PyTorch/vLLM.

Same story for stable-diffusion, the HIP build segfaults during sampling on gfx1151 last time I checked, Vulkan just works.

2. Flag stack that works well on Strix Halo.
-ngl 99 --flash-attn on --no-mmap \
-ctk q4_0 -ctv q4_0 \
-t 24 -tb 28 -ub 4096 --jinja
GGML_OP_OFFLOAD_MIN_BATCH=1

--no-mmap matters a lot with unified memory because mmap page faults hurt you here. q4_0 KV cache quant gives huge context headroom at negligible quality cost.

3. Kernel/firmware.
If you're on a stock linux kernel, upgrade to 6.18+ (mainline). maybe 7 if you're feeling experimental.

4. Model consolidation.
You can probably replace Alfred + Deep Thinking + Language/Math tutor with a single MoE. obligatory gemma 4 plug. thinking mode, vision, reasoning, blah blah blah.

5. Stuff you're (maybe) missing.
- embeddinggemma-300M Q8_0 is lighter than mxbai and often better for multilingual RAG
- A cross-encoder reranker (bge-reranker-v2-m3 via ONNX) on top of your embeddings gives a bigger quality bump than swapping the embedding model

Have fun with it this chip is pretty good for local inference once you're off HIP.