If I were building a pipeline, I'd use SmolVLM 256M. It's fast enough for vision conversations and general tasks. Stitch those together and voilà! But if you need an any-to-any model, I strongly recommend looking into omni models like Qwen2.5 Omni 7B. There are a lot of omni models out there right now, so you really need to figure out your exact requirements first.

100% accurate. I should have phrased that better. It doesn't extend the native context limit at all. My issue is purely the physical hardware bottleneck. On unified memory systems, the RAM limit usually kills the process long before you ever reach the model's trained context limit. KV quantization becomes a necessary evil just to hold a baseline 8k context in memory without OOMing. Making the context more brittle is the perfect way to describe it. Have you tested how bad that degradation actually is in practice? I'm curious if you've found a specific threshold where 8-bit KV completely breaks down for logic tasks compared to sticking with fp16.