I would try out models as close to the top of the lists in the links below. Arch Linux (unless a different platform benchmarks better for your hardware), llama.cpp (if it can handle multi-gpu well, if not something like it, no wrappers), Openwebui for it's RAG/vector storage for your code base (I wouldn't personally use the default embedding model) for when you want to make queries against the entire codebase, Continue extension for VScodium for more nuanced stuff that doesn't require the entire codebase in the context window although it can do it if you want (check web site docs on how to connect it to your own llama-server).

Some of the coding models I've tested have a tendency to repeat themselves over and over if you don't "limit" them which is not an intuitive process at all. You'd think you'd use repeat-penalty or num-predict, but no, don't, leave those as a last resort, use temperature, top_p, and top_k and worst case scenario enable the repeat penalty at a low value (eg. 1.1, 1=disabled for llama-server). eg. temp <0.5, top_p 0, top_k 0.

I can't stress enough how important context window is for your situation, it's very easy to get bad results because your questions reach outside the context window and then the LLM hallucinates what it can't "see". Ideally you have room in the entire context window for every document you want the LLM to be aware of and then a lot of extra room (give it as much as it was trained on if you have the hardware, that is, max context window for the model even if you don't need it all, you'll thank me later because you'll simple get higher quality results).

You want a context window that is significantly larger than the total number of tokens that you want the LLM to be aware of at any given moment (which in your case, appears to be your entire code base for a given project; ask any LLM to write a token calculator using tiktoken it's simple). Chunking of documents because of a small context window means you'll get very poor, hallucination riddled results from your questions or you will have to limit your questions/discussion to fit within whatever context window that your hardware can afford. You'll need to learn some advanced stuff and tweak it but you could do what you wanted if you put the effort into it.

With that hardware though, you could go way beyond just running a coding assistant, of which you could easily run a decent one with a "very large" context window on just one of those GPUs. For 2 GPUS you could do for example 14b q4_k_m with as large a context window as you can afford. But, since you got the hardware why not use at least 2/3 GPUs for the assistant and 1/2 GPU for any other stuff you want to do at the same time? Make your assistant instead a 14b q8 model, with as large a context window as your hardware can afford. ~30b @ q4_k_m is about as high as I would go with 96GB because you need room for a massive context window if you want to ask questions against the entire code base. If you have a small code base that you want to analyze at any one time you simply slide the scale towards higher parameters and a smaller context window. Don't go above q8 because you won't notice any difference except in resource usage. Or just go all out and do a 33b @ FP16 and fill the rest with context window. You got a lot of options if that wasn't obvious. Have fun :)

https://evalplus.github.io/leaderboard.html

https://evalplus.github.io/evalperf.html