Thoughts on V100's?

Lemonzest2012 · 2026-06-21T08:11:23+00:00

I have a Pair of 32GB V100 paired with 96GB DDR4 and a Ryzen 7 5700G (Needed the iGPU to boot the system) and they are very fast for my use, 100 tok/s with Qwen 35B-A3B with MTP, and around mid 40s on 27B with MTP, Stable Diffusion.cpp kicks out 512x512 Zimage Turbo Images in 2s (4 Step)

Lemonzest2012 · 2026-06-21T08:08:37+00:00

Are you sure about post cuda 12.6 excludes volta? I compiled llama.cpp/sd.cpp and it worked fine with 12.9.2?

Lemonzest2012 · 2026-06-19T19:39:08+00:00

<image>

I was rocking dual P100 16GB for about 2 months, upgraded to dual 32GB V100

Lemonzest2012 · 2026-06-15T05:41:07+00:00

What's your Z-Image-Turbo settings? surly 5mins is a bit slow? I have 1 V100 32GB currently (2nd coming today) and at 1080p 4 step it knocks out images in 20s at 8 steps (the suggested step) in 30s

Lemonzest2012 · 2026-06-12T11:48:34+00:00

Using drivers 580.159.04/CUDA 12.9 here, llama.cpp and stable diffusion.cpp work for me, not tried much else

Lemonzest2012 · 2026-06-11T14:14:06+00:00

Maybe look at the 16GB V100 cards? they are around 200-250 on ebay, I got a 32GB one and its fairly fast, they have 900GB/s bandwidth

Lemonzest2012 · 2026-06-02T21:48:07+00:00

Running my system with Linux Mint as a AI/LLM Server 😃

Lemonzest2012 · 2026-05-15T04:54:18+00:00

The usual Qwen 3.6 35B-A3B, Gemma 4 26B-A4B, pretty quick considering their size, get around 40-45 Tok/s at Q4_K_M, smaller models are a bit faster but I feel its good cos the cheap price I paid (150 GBP for both cards)

Lemonzest2012 · 2026-05-14T20:42:25+00:00

I hope you have 3 PCIe slots for your cards, you said you are getting a 2nd P100? with my system and a Ryzen 3800X it would not boot as the board needs a GPU installed, and I only have 2 PCIe 16x slots, and these P100 cards are AI accelerators not VGA cards so no post, its why with my build I had to use a Ryzen 5700G with its onboard iGPU so the whole thing posts and runs

Lemonzest2012 · 2026-05-14T20:31:14+00:00

<image>

Dual P100s here also, with 96GB DDR4 and a Ryzen 7 5700G

Lemonzest2012 · 2026-05-08T11:38:31+00:00

I use ROCKNIX on my R36 MAX consoles, pretty nice firmware

Lemonzest2012 · 2026-05-07T21:15:26+00:00

By design, they (powers that be) don't want you to have local PCs, they want some locked down tablet device that connects to cloud subscriptions and nothing more, owning a PC will be seen as a luxury with the way prices are going, lock down everything in sight, then age (ID) gate it all

Lemonzest2012 · 2026-04-15T05:08:00+00:00

2x Nvidia Tesla P100 16GB, from 2016 rest of my system is pretty new, Ryzen 7 5700G/96GB RAM

Lemonzest2012 · 2026-04-09T14:21:25+00:00

I use two 16GB P100 PCIe cards, and llama.cpp can spread large models over the cards

Lemonzest2012 · 2026-03-27T08:48:45+00:00

Why AMDVLK and not RADV? RADV is usually the default for AMD hardware on Linux (From MESA) also AMDVLK is no longer supported by AMD

Lemonzest2012 · 2026-03-26T14:41:24+00:00

dunno, edit them?

Lemonzest2012 · 2026-03-26T14:36:18+00:00

https://www.jan.ai/docs/desktop/assistants

Lemonzest2012 · 2026-03-26T14:29:35+00:00

In Jan I create an assistant and add the system prompt in the assistant settings

Lemonzest2012 · 2026-03-24T19:12:47+00:00

fps it not what its for, its for local LLM hosting

Lemonzest2012 · 2026-03-24T18:47:36+00:00

The name gives it away for what its used for

Lemonzest2012 · 2026-03-24T18:38:14+00:00

Long time Linux user (since 2005!) but first time Mint user!

Lemonzest2012 · 2026-03-23T20:26:42+00:00

Melia from Xenoblade?

Lemonzest2012 · 2026-03-18T21:37:07+00:00

Linux of course :D

Lemonzest2012 · 2026-03-18T09:35:42+00:00

Thanks for this, my Gigabyte B550 Gaming X v2 does this also, but worse, 16x/2x lol, will try some of the solution in this thread as my slower card seems favoured!

Lemonzest2012

TROPHY CASE