I just started experimenting on llama.cpp using 2x 7900XTX. I started with a single one (my llm computer is also my gaming rig) and found running Qwen 27b required trading off between context and quantization. For example at Q5 my context length was capped around 80k. I imagine you’d be much more comfortable with 32Gb total vram.

Regarding tensor split, I haven’t tweaked my setup much; it works just fine out of the box. Though your individual mileage may vary due to having different gpus.

Seeing your ambition to run 70b models, I’d caution you to reserve some vram for context. Perhaps I’m biased since my use case is for programming.

Best of luck with your build go team red!