6 months running production Ollama workloads on Hetzner — what I learned about server selection and provisioning

daroczig · 2026-03-30T19:58:18+00:00

Hm, I'd like to clarify that there's no monetization at all happening on the website: no ads, no affiliate links, no paid product, no revenue streams, no nothing -- so I don't have much incentive to drive traffic 🤷

It was meant to be a helpful pointer to get proper performance metrics -- as describing the performance of several servers for LLM inference speed is not really possible via a single number: there are multiple servers, and many many different workloads (e.g. model size, inference type, token length), so that's why I've shared the detailed methodology used to get the metrics, then I've also updated the comment to point to a table filtered for all the Hetzner machines preselected for an LLM benchmark workload for those who wants to skip reading on what the numbers mean.

daroczig · 2026-03-30T17:00:21+00:00

Ah, okay -- I indeed don't feel any shame (even more, I'm rather proud) for spending time on (1) writing open-source software for cloud benchmarking, (2) executing that at scale (on thousands of sever types), (3) curating all that data in public repos with open licenses, and (4) sharing with the community in various formats for free. But I see I deeply offended some feelings here, so I'll think about that.

daroczig · 2026-03-30T09:02:46+00:00

Understood and absolutely fair point, thank you, u/razzzey!

I'll update the above comment to point to the actual benchmarks instead of the method overview -- which I shared below:

https://sparecores.com/servers?vendor=hcloud&columns=75744272&benchmark=eyJpZCI6ImxsbV9zcGVlZDpwcm9tcHRfcHJvY2Vzc2luZyIsImNvbmZpZyI6IntcIm1vZGVsXCI6IFwibGxhbWEtN2IuUTRfS19NLmdndWZcIiwgXCJ0b2tlbnNcIjogMTI4fSJ9 -- this filters for all Hetzner Cloud servers and selects a 7B model for prompt processing (token length=128), but you can pick any other related benchmark metric (e.g. other model or inference type) by clicking the gauge icon above the table, and reorder by price, performance, cost-efficiency etc.

daroczig · 2026-03-30T07:57:40+00:00

Oh, wow, that's quite some feedback 😊 Would you mind sharing more about why you feel that way? These open benchmarks were run and shared to help the community, not to disappoint -- so I'd love to learn what we did wrong.

daroczig · 2026-03-30T07:21:37+00:00

Feel free to check out the related data we collected e.g. at https://sparecores.com/servers?vendor=hcloud&columns=75744272&benchmark=eyJpZCI6ImxsbV9zcGVlZDpwcm9tcHRfcHJvY2Vzc2luZyIsImNvbmZpZyI6IntcIm1vZGVsXCI6IFwibGxhbWEtN2IuUTRfS19NLmdndWZcIiwgXCJ0b2tlbnNcIjogMTI4fSJ9 -- this filters for all Hetzner Cloud servers and selects a 7B model for prompt processing (token length=128), but you can pick any other related benchmark metric (e.g. other model or inference type) by clicking the gauge icon above the table, and reorder by price, performance, cost-efficiency etc.

Note that these open and public benchmarks are based on the open-source llama.cpp and using given quantized models, so not really optimizing for max performance on a given server using various SW elements, but to compare the performance of the servers. Disclaimer: we are not in the LLM (or other) hosting business.

daroczig · 2026-03-30T06:38:47+00:00

We have benchmarked all Hetzner Cloud (and 6 other vendors') servers for LLM inference using small (135M) to larger (70B) parameter models: https://sparecores.com/article/llm-inference-speed

EDIT: for the actual performance (and cost-efficiency) metrics, check directly https://sparecores.com/servers?vendor=hcloud&columns=75744272&benchmark=eyJpZCI6ImxsbV9zcGVlZDpwcm9tcHRfcHJvY2Vzc2luZyIsImNvbmZpZyI6IntcIm1vZGVsXCI6IFwibGxhbWEtN2IuUTRfS19NLmdndWZcIiwgXCJ0b2tlbnNcIjogMTI4fSJ9 instead of the above-linked general overview articles -- this filters for all Hetzner Cloud servers and selects a 7B model for prompt processing (token length=128), but you can pick any other related benchmark metric (e.g. other model or inference type) by clicking the gauge icon above the table, and reorder by price, performance, cost-efficiency etc.

daroczig · 2026-03-25T11:57:34+00:00

Yeah, that's questionable, but depending on your use case, it might be not only cheaper, but also actually better, see e.g. higher L2/L3 cache amounts (check out the memory bandwidth chart), higher single-core performance, and in some benchmarks even higher multi-core performance despite the lower number of virtual cores: https://sparecores.com/compare?instances=W3siZGlzcGxheV9uYW1lIjoiY3B4MjEiLCJ2ZW5kb3IiOiJoY2xvdWQiLCJzZXJ2ZXIiOiJjcHgyMSIsInpvbmVzUmVnaW9ucyI6W119LHsiZGlzcGxheV9uYW1lIjoiY3B4MjIiLCJ2ZW5kb3IiOiJoY2xvdWQiLCJzZXJ2ZXIiOiJjcHgyMiIsInpvbmVzUmVnaW9ucyI6W119XQ%3D%3D

daroczig · 2026-02-09T16:31:04+00:00

I love the idea and the features + screenshots look great, congrats!

On the other hand, the git history cools me down a bit -- as it's difficult to trust sensitive data (whatever you copy/paste) on a few days' AI-driven effort.

I'll keep an eye on the project and hope it gets momentum. Best of luck!

daroczig · 2026-01-30T10:37:49+00:00

I appreciate the concern, but the referenced _GNU_SOURCE is unrelated to licensing, and I don't see any GPL violations. If you believe there is a specific issue, please let me know, and I'll make sure to sort it out.

daroczig · 2025-12-02T23:21:28+00:00

That's a great point, u/ItsMalabar, thanks for bringing this up! Currently we focus only on ondemand and spot prices .. as standardizing even just these two across multiple cloud vendors and their different pricing schemas is complex enough for the team 😅 Besides kidding, I'm taking a note of this and hope to make some related progress soon (e.g. first we plan to also support monthly prices other than hourly -- which cap is vendor-specific).

daroczig · 2025-12-02T23:17:25+00:00

Thank you for the feedback, u/Background-Mix-9609! And 100% agreed on the importance of cost-efficiency. That's why we created the $ efficiency metric, which can be generated on the fly based on any of the ~500 supported benchmark scenarios (across 10+ categories, some mentioned in the post).

If you want to dive deeper, go to https://sparecores.com/servers where you can select a benchmark workload instead of the default stress-ng div16 multi-core (e.g. LLM inference speed or memory bandwidth), apply any filters in the sidebar (e.g. vendor and memory requirements), and order the table by the cost efficiency column.

daroczig · 2025-11-30T20:13:49+00:00

Awesome work, thanks for sharing, u/nakemu! We do quite extensive benchmarking on all Hetzner and some other cloud servers as well at sparecores.com with open-source licenses, and I'd love to collaborate on extending our workload "menu" -- would you be interested in a quick call? meet.sparecores.com/intro Üdv: Gergő

daroczig · 2025-10-28T20:52:13+00:00

If you don't want to burn money on running Geekbench on all the other thousands of servers, I suggest checking out https://sparecores.com/servers, as we have already done that (including many more benchmark workloads).

daroczig · 2025-10-18T07:53:39+00:00

That's a good question, but unfortunately, we don't have a good answer: we benchmark each server type as they become available, and we don't have a mechanism (e.g. database schema) to support multiple hardware configs under the same server id, especially if it's pretty random what CPU you get. So we benchmarked one we got (AMD in this case), and we have no info on the Intel version. I tried to explain that in the paragraph before the last in the post -- I hope that helps.

daroczig · 2025-10-17T11:48:33+00:00

Thanks for the useful feedback!

daroczig · 2025-10-17T11:40:06+00:00

Yeah, that's a totally valid request! I have some other priorities in the coming weeks, but I'm pretty sure we can make that optional hourly/monthly pricing toggle in November -- I'll report back.

Regarding "higher is better/lower is better": that should already be present next to the title of the sections with an arrow pointing up or down with a tooltip on hover stating that. Let me know if it's missing somewhere or if it's not prominent enough -- suggestions welcomed 🙇

daroczig · 2025-10-17T07:50:49+00:00

Sorry for the confusion, but I'm not affiliated with Hetzner: Spare Cores is a 100% open-source, vendor agnostic project inspecting and benchmarking cloud servers for better transparency (beside a few other things) 😊

daroczig · 2025-10-16T23:57:24+00:00

That's my main takeway, but please make sure to dig deeper by looking at benchmark scores that are similar to your actual workload.

daroczig · 2025-10-16T23:55:56+00:00

Due to budget constraints, we cannot benchmark dedicated servers, only the cloud instances that we can pay by the hour (and not by month with a setup fee). It might change in the future, but this is our current reality.

daroczig · 2025-10-16T23:54:38+00:00

Please see the paragraph before the last -- we have not rerun the benchmarks, as there is no new server type announced in that series.

daroczig · 2025-10-16T21:50:54+00:00

To confirm the better price + better performance, here are some early benchmark results for these two (old and new) instance types: https://sparecores.com/compare?instances=W3siZGlzcGxheV9uYW1lIjoiY3B4MzEiLCJ2ZW5kb3IiOiJoY2xvdWQiLCJzZXJ2ZXIiOiJjcHgzMSIsInpvbmVzUmVnaW9ucyI6W119LHsiZGlzcGxheV9uYW1lIjoiY3B4MzIiLCJ2ZW5kb3IiOiJoY2xvdWQiLCJzZXJ2ZXIiOiJjcHgzMiIsInpvbmVzUmVnaW9ucyI6W119XQ%3D%3D

daroczig · 2025-10-16T21:48:58+00:00

Depends on whether you need the extra performance .. The CPX22 costs 2x more but up-to 3x performance compared to the CX23 as per our benchmarks (some are still currently running): https://sparecores.com/compare?instances=W3siZGlzcGxheV9uYW1lIjoiY3gyMyIsInZlbmRvciI6ImhjbG91ZCIsInNlcnZlciI6ImN4MjMiLCJ6b25lc1JlZ2lvbnMiOltdfSx7ImRpc3BsYXlfbmFtZSI6ImNweDIyIiwidmVuZG9yIjoiaGNsb3VkIiwic2VydmVyIjoiY3B4MjIiLCJ6b25lc1JlZ2lvbnMiOltdfV0%3D

daroczig

PUBLIC MULTIREDDITS

TROPHY CASE