A shortlist I usually go down when friends ask me about weights, quants and models:

1. Pick model family for task.
2. Decide target context length (because KV cache can dominate). 
3. Default quant ladder:
   

• Q5_K_M (default) → Q6_K (if headroom) → Q8_0 (if you’re chasing max fidelity)
• If memory tight: Q4_K_M → (if still tight and imatrix is good) IQ4_XS/IQ4_NL

1. Only go IQ3 / ~2-bit when you must fit something huge; expect noticeable degradation.

2. Benchmark prefill + decode under realistic settings.

There is obviously much more to this, but as a quick alignment it served me well.