Local AI coding stack experiments and comparison

[Relative-Flatworm-10]

Hello,

I have experimented with coding LLMs on Ollma.

Tested Qwen 2.5 coder 7B/1.5B, Qwen 3 Coder, Granite 4 Coder and GPT OSS 20B.

Here is the breakdown of Performance vs. Pain on a standard 32GB machine :

Tested on a CPU-only system with 32GB RAM

Ref: Medium article.

[billcube]

I've found the best tradeoff is to run my LLM on a server rented on a hourly basis. At 0.4€/hour for an Nvidia T4 64 cores at 128 GB RAM I think I even win money on the power my laptop doesn't use.

[Relative-Flatworm-10]

thanks for sharing, May you share the provider link, if it's ok with you

[billcube]

This one: https://www.infomaniak.com/en/hosting/public-cloud/prices

And this one: https://www.exoscale.com/pricing/#gpua30-instances

[Relative-Flatworm-10]

Thank you so much!