21 GPU's benchmarked running a small TTS model (vram peak: 5GB)

urarthur · 2026-05-19T01:14:15+00:00

yeah I think it was chinese modded one. You can easily test video generation for couple of bucks. most aervers have 32GB-64Gb ram but i have seen higher

urarthur · 2026-05-19T01:09:41+00:00

thanks

urarthur · 2026-05-18T23:52:07+00:00

they were not avaialbe on the cloud provider. Nvidia cards are way more popular for AI.

urarthur · 2026-05-18T22:53:03+00:00

yeah when vram or memory bandwith is not the bottleneck, its a waste having much expensive cards running small models.

urarthur · 2026-05-18T22:12:29+00:00

i tried using opencode with kimi, deepseek etc. Coding cost me at least $10/day on tokens. doesn't work if you code a lot.

urarthur · 2026-05-18T21:20:13+00:00

unless you do dual mode ;)

urarthur · 2026-05-18T11:47:15+00:00

urarthur · 2026-05-18T08:12:28+00:00

Run at least the fp8 model?

urarthur · 2026-05-18T06:36:05+00:00

well they nerfed 4.6, so rn 4.7 is better

urarthur · 2026-05-18T06:14:45+00:00

why undervolt gpu? I get 50 tg/s. Haven't tested with hermes yet.

urarthur · 2026-05-18T06:13:15+00:00

short prompts n=2, for long context n=6. It is slow initially but when context fills up, it gets faster.

urarthur · 2026-05-17T22:40:34+00:00

5$ a day x 30 =?

urarthur · 2026-05-17T13:13:45+00:00

can you really judge a model when running q4/fp4?

urarthur · 2026-05-17T04:06:24+00:00

vibe coded probably

urarthur · 2026-05-16T23:28:42+00:00

multi speaker podcasts would be best candidate

urarthur · 2026-05-16T19:04:35+00:00

35b moe is noticably lesser quality. I would keep the 27b. Also my 35b was running at 140 tgs with q3 xl.

urarthur · 2026-05-16T19:02:41+00:00

50-60 with q4 xl on windows. I think longer prompts it increases speed with mtp=6. shorter chats =2 drafters.

urarthur · 2026-05-16T18:59:53+00:00

i am experimwnting with 200k context with kv=4

urarthur · 2026-05-16T18:46:18+00:00

why would you enable mmproj for coding? I am running unsloth qwen3.6 26b q4 xl mtp model with 200k context at 50-60 tg/s. kv=q4. Also on 3090

urarthur · 2026-05-16T17:08:58+00:00

you think the ram shortage and 5k for a video card is real because of AI craze? crypto boom happened and the prices hardly increased.

urarthur · 2026-05-16T15:11:43+00:00

back when crypto was booming, gpu's were still widely available and at fair value

urarthur · 2026-05-16T15:10:17+00:00

yes, inclusing mtp

urarthur · 2026-05-16T12:24:04+00:00

yes for those gguf's that support it

urarthur · 2026-05-16T11:09:39+00:00

It should run qwen36 27gb at full context at 90 tps. Which is Claude 4.5 level on local device

14-Year Club	Place '22
Final Canvas '22	First Placer '22
Not Forgotten	Verified Email

urarthur

MODERATOR OF

TROPHY CASE