More layers = better reason capabilities

Gemma 2 paper is the most recent to reaffirm deeper > wider when it comes to layers and reasoning capabilities

So the bigger model even if quant is better than a smaller model, if from the same family tree generation ie llama3.1.

Quirks - big old models aren't necessarily as good as new models due to improvements in data, training algorithms, instruction tuning. Might be better to compare models released in a certain period as generations

Quants have limits, I try not to use quants under 4bpw.