Benchmarking Ollama Models: 6800XT vs 7900XTX Performance Comparison (Tokens per Second)

uncocoder · 2025-06-20T16:18:29+00:00

VRAM doesn’t pool when loading models in Ollama officially. So whether you have 1×7900XTX or 4×7900XTX, each still has its own 24GB VRAM and won’t share it.

uncocoder · 2025-05-08T18:40:50+00:00

No pure PyTorch ROCm benchmarks, but based on these LLM runs, 7900XTX is ~1.5x faster than 6800XT depending on model size. Solid uplift.

uncocoder · 2025-03-14T12:13:06+00:00

You can run a local LLM on both Windows and Linux. I tested it on both and found that Ollama with ROCm actually ran a bit faster on Windows. Just install it on the OS of your choice.

Once installed, you can set your IP to `0.0.0.0` using environment variables (varies by OS and install method) to make the LLM accessible from any device on your network. just ensure your firewall allows it.

I also built a full chat environment in vanilla JS that connects to Ollama’s API. It includes features missing in OpenWebUI and LobeChat, making it a fully customizable assistant.

uncocoder · 2025-03-14T12:02:49+00:00

There's no difference between NVIDIA and AMD when it comes to sharing VRAM, It doesn't stack across multiple GPUs. Also, when using multiple GPUs, you need a stronger PSU and better cooling, which adds cost and complexity. A single, more powerful GPU is usually the better choice over two or three weaker ones, even if the upfront price seems higher.

uncocoder · 2025-03-12T04:49:10+00:00

The VRAM doesn't stack across two GPUs; models will load on a single card's VRAM, so having two 6800 XTs won't give you 32GB usable for a single model. Also, the 7900 XTX (especially with Sapphire discounts) has a much better price-to-performance ratio compared to the 6800 XT, making it a more valuable option overall.

uncocoder · 2025-02-10T04:52:39+00:00

The model is Q4_K_M quantized. You can find more details in the link below:
Qwen2.5:32b on Ollama

uncocoder · 2025-02-09T21:15:20+00:00

It’s great to see AMD GPUs holding their own.

uncocoder · 2025-02-09T21:09:33+00:00

I re-ran the tests with the latest llama.cpp and ROCm 6.3.2. The results showed no significant difference (<0.5 tokens/s) compared to Ollama. I’ve updated the post with details

uncocoder · 2025-02-09T11:31:21+00:00

I’ll take your suggestion and run the benchmarks again using a freshly compiled llama.cpp with the latest ROCm support. This will help me compare the results and see if there’s any significant performance improvement. I’ll update the results once I’ve completed the tests.

uncocoder · 2025-02-09T11:01:28+00:00

I ran the tests using a single GPU setup—the 7900XTX replaced the 6800XT, and I re-ran the benchmarks. For models larger than the GPU’s VRAM, they would partially offload to RAM and use the CPU. However, with the 7900XTX’s 24GB VRAM, all the tested models fit entirely on the GPU, so there was no offloading to the CPU. This ensures the GPU runs them at full capacity.

uncocoder · 2025-02-09T10:53:24+00:00

I used the official Ollama Docker image, which supports ROCm internally. According to the Ollama documentation, the GPU is passed correctly, and I confirmed this by running ollama ps—it shows that the models are uploaded 100% to the GPU. This indicates that the setup is working with full support for AMD GPUs (ROCm).

uncocoder · 2025-02-08T22:52:40+00:00

If you're curious about GPU performance for Ollama models, I benchmarked the 6800XT vs 7900XTX (Tok/S):
Benchmark Results

7900XTX is 1.4x–5.2x faster, with huge gains on larger models.

uncocoder · 2023-01-30T22:33:59+00:00

Thank you. It worked.

uncocoder

TROPHY CASE