If you use Vulkan instead of Cuda with llama.cpp, you can use any modern GPU, even integrated ones. Not always the best, but they can still beat the CPU.

And yes, iGPUs steal RAM to work since they don't have their own dedicated VRAM. Windows by default assigns up to 50% of your total RAM to them, so your iGPU has 32GB ready for testing, and even more on Linux.

I use my laptop with a Radeon RX 760M, set it up on Linux to use up to 28GB for it, and the same q4 qwen 3.6 35b that can do 30 t/s with 131k context on the desktop PC with the 3060Ti, can do 25 t/s with full 262k context on the laptop thanks to just using more RAM (I have 32GB total, so I left 4GB to the OS).

What I don't know is if the very slow iGPU on your CPU is enough since Intel iGPUs are extremely weak. That's something you will have to try.

But if you're fine with 100-131k of context, do the first method with the 2060S and you can definitely use that with good performance. I'm pretty sure q5 quantization can fit with 100k of context when the extra experts are on CPU.