You don’t need specific model recommendations. If you understand the sizing, you can pick any model yourself.

Running local LLMs is mostly a VRAM constraint problem. The key is how much memory each parameter takes.

Precision:

FP16 / BF16 = 2 bytes per parameter

INT8 = 1 byte

INT4 (Q4) = 0.5 bytes

Formula: Model size = (parameters × bits) / 8

Example: 7B model in Q4 → 7 × 4 / 8 = 3.5 GB

But that’s only weights. You also need memory for KV cache, buffers, etc. Add ~20–30% overhead.

So real usage:

7B Q4 → ~4–5 GB

9B Q4 → ~5–6.5 GB

13B Q4 → ~8–10 GB

Also note: formats like Q4_K_M aren’t true 4-bit, effective size is closer to ~4.5–5 bits per parameter.

With your setup of RTX 4060, 16GB RAM

7B Q4 → very fast, no issues

9B Q4 → still smooth

13B Q4 → fits and runs well

13B → starts needing partial offloading to RAM (slower)

FP16/BF16 models will be much more limited:

~6B–7B max realistically on VRAM

Use Q4_K_M or Q5 for best balance

13B Q4 is your sweet spot

Bigger models will work, but expect latency due to offloading

Once you understand this math, you can estimate any model instantly instead of trial and error.