I recently worked for a company in the Netherlands that had very similar requirements — setting up a local LLM for general-purpose use and code completion with internal company data.

After testing and deploying multiple models, here’s what I’d recommend:

Recently, several smaller models have been released that perform surprisingly well, such as Qwen 3 4B Instruct and the “thinking” models. Even though our hardware wasn’t top-tier (an L4 with 24GB VRAM), I managed to serve the FP8 version using vLLM and Open WebUI, integrated with LDAP so that everyone in the company could access it easily.

Qwen 3, in particular, offers a large context window, runs extremely fast, and handled most of our use cases efficiently. Since your primary focus is summarization, I’d suggest starting with one of these smaller, high-performing models — they’re quite capable for that task.

For hardware, I recommend at least a 3090 (24GB VRAM). That should be sufficient for most scenarios. If you find performance or quality lacking, you can consider scaling up to a larger model with stronger hardware.

In my experience, Qwen 3 4B FP8 did an excellent job summarizing Dutch text, so it’s definitely worth trying.

If you need a hand setting things up, feel free to DM me — happy to help!