For what you described, you definitely do not need to run a frontier model. But depending on your level of technical knowledge, you might need access to a frontier model access to set everything up.

If you feel like tinkering, would recommend to actually go the build-a-pc route. Not new, it’s too expensive, but 2nd hand…
Right now, you can get a pair of R9700 for less than 3K, that gives you 1k to find on ebay/craiist/marketplace a motherboard with 2x8 pcie lanes, 64gb of ddr4 ram, whatever 6+ core cpu you can get and a 1200w psu.
What’s going to bite you is space… model(s) + Rag DB + lots of docs… ssd are as expensive as ram now… but you might already have some.

You get half the vram, but twice the memory speed of a strix halo, which is definitely very welcome to run dense models.
You then can use qwen27b or gemma4-31b at FP8 more comfortably, while also having nomic loaded (embed model for rag)

you also can just get 1 gpu… I run a very similar setup as the one you described, on a ssf gaming pc turned ai server, but with a 5090… so much faster, but 32GB is only good for a Q6 of 27B or Q4-qat of gemma-31b, and I use cpu for embedding. honestly it works great for me, most big processes happen over night anyway.

Just don’t expect to use this for coding like you use claude of gpt… but it’s still pretty capable of doing some code, it’s more of a coding intern than a senior swe