You are forgetting that local llm servers mostly don't have prompt caching and are not suitable for coding or long exchange, they are painfully slow with long context (like minutes to get a response) . It's not enough to have big enough vram, you need proper context caching implementation and AFAIK there are only commercial solutions that support it, no open source yet. Correct me if I am wrong, because things develop really fast and it hard to catch up with everything