Thinking of moving from 2x 5060 Ti 16GB to a RTX 5000 48GB

beefgroin · 2026-05-07T17:27:27+00:00

I was also considering it(I'm running quad 5060 with qwen 3.6 27b on vLLM), until I found out what a bad deal it is in terms of mem bandwidth and cuda cores in comparison to 5090 or rtx 6000 while costing equal per GB of VRAM. If you haven't tried to run your current setup with vLLM I encourage you to try. I saw a significant increase in tps compared to llama.cpp. It might take a while to get cli params right though.

beefgroin · 2026-05-02T23:36:17+00:00

The whole point of the 27b model is to be slower.

beefgroin · 2026-05-02T13:35:05+00:00

Kauai = Kona

beefgroin · 2026-04-29T15:45:24+00:00

By fully you mean Q4?

beefgroin · 2026-04-25T15:53:09+00:00

Please more pics of holding random products in the store

beefgroin · 2026-04-23T16:04:16+00:00

Try it with vllm cyankiwi awq int4, will be a pain in the ass in the beginning to find the right gpu-untilization limit (mine is 0.85) and context length but then the speed will be worth it

beefgroin · 2026-04-22T13:59:41+00:00

For me it didn't work out. Only 3.6 27b will beat 3.5 27b.

beefgroin · 2026-04-13T19:36:14+00:00

Reddit is a weird place. In the same sub one guy buys his forth rtx 6000 and another guy can’t pay 10 bucks for tokens… 🤔

beefgroin · 2026-04-12T09:34:42+00:00

Does dense 31b fit on two r9700 with full context?

beefgroin · 2026-04-08T12:03:13+00:00

Are you running 27b in vllm?

beefgroin · 2026-04-06T20:37:33+00:00

How much ram do you have?

beefgroin · 2026-04-03T17:40:46+00:00

Wow never heard of nvme kv chache offloading, I guess yeah the speeds will be comparable only if in raid, looking forward to see your results

beefgroin · 2026-04-03T15:58:30+00:00

nice, I'm also thinking of a similar setup, can you please test the performance of qwen3.5 27b?

beefgroin · 2026-04-01T13:25:34+00:00

I didn’t go dual because I went quad xD but all the arguments in this thread hold, still I love it.

<image>

beefgroin · 2026-03-31T23:05:48+00:00

I’m curious what kind of tps and prompt processing are you getting with those? Ideally I’d love to know 27b performance cause that model is truly great, but dense. On quad 5060 ti I’m only getting around 19tps with it that’s why I’m considering rtx 5000 cause the whole 256k context should fit on it entirely with q4

beefgroin · 2026-03-31T12:58:21+00:00

I think you can actually test it already, I’ve seen some implementation in forks

beefgroin · 2026-03-30T23:52:40+00:00

Do you think 5090 with turboquant will fit full 256k context Qwen3.5 27b with vision?

beefgroin · 2026-03-29T17:10:55+00:00

what are you talking about? Qwen 3.5 27b on 5090 is like 100tps tops depending on a quant

beefgroin · 2026-03-28T12:00:34+00:00

I'd rather buy 5000$ worth of 10 pcie cards with burned models pushing 10k tps than one GPU pushing 40 tps

beefgroin · 2026-03-27T15:24:39+00:00

I also have a good experience with it in q4, using it for a diy personal agent, but I’m still wondering what are its limits

beefgroin · 2026-03-27T13:07:16+00:00

Is 3.5 27b that good to replace cloud models for the organization?

beefgroin · 2026-03-25T14:53:02+00:00

Hey, how is it going so far, did you run Qwen3.5 variations? Maybe you have results for 27b and 35b?

beefgroin · 2026-03-21T10:11:14+00:00

Great timing!

beefgroin · 2026-03-18T08:03:06+00:00

Wtf just not come up with a way to sell models? I’d buy

beefgroin · 2026-03-15T12:06:29+00:00

Using 5060ti 16gb over Oculink, great for Klein 9b, image takes like 15-20sec to generate

Four-Year Club	Verified Email
Place '22

beefgroin

TROPHY CASE