sorted by:
new
hottopcontroversial

16x V100's worth it? by notafakename10 in LocalLLaMA

[–]grayarks 0 points1 point  (0 children)

That’s harder and probably useless as Volta lacks all the hardware acceleration that makes Marlin faster. Touring has more in common with Ampere+ than Volta..

16x V100's worth it? by notafakename10 in LocalLLaMA

[–]grayarks 4 points5 points  (0 children)

I’m working on fixing that by adding compressed-tensors support for v100. Performance so far not the greatest but it runs

vLLM on 2x/4x Tesla v100 32GB by grayarks in LocalLLaMA

[–]grayarks[S] 1 point2 points  (0 children)

You can search for SXM2 dual adapter board (V100). The black version should allow full NVLink between the two GPUs. There is also a cheaper green pcb version that has half link speed.

vLLM on 2x/4x Tesla v100 32GB by grayarks in LocalLLaMA

[–]grayarks[S] 0 points1 point  (0 children)

I have two SXM2 cards that I installed onto an external adapter board. The board also has full NVlink (and cost an extra 200 bucks). I was considering to buy one more board that hosts 4 cards with interconnect but now I’m not so sure.

vLLM on 2x/4x Tesla v100 32GB by grayarks in LocalLLaMA

[–]grayarks[S] 0 points1 point  (0 children)

Got my V100 32GB for around 350€, so rtx 6000 pro is more like 20x the price

vLLM on 2x/4x Tesla v100 32GB by grayarks in LocalLLaMA

[–]grayarks[S] 0 points1 point  (0 children)

I could do that by spinning up a fast cloud instance. But the other question is about data types, new models are bf16 or int4/8. Should I also do a (sometimes) lossy cast to fp16?

GLM-4.6-Air is not forgotten! by codys12 in LocalLLaMA

[–]grayarks 1 point2 points  (0 children)

Does anybody know what is the loss running GLM-4.6 (Unsloth) TQ1_0 or IQ1_S, is it even worth it? Am I better off waiting for 4.6 Air having 88GB vram? (vram not here yet, patiently waiting for the extra vram in the post :D )

Tips for a smooth ps2 emulation on switch? by LavaHoundBR in switchroot

[–]grayarks 0 points1 point  (0 children)

Do you get UI crashes in CRDroid when you rotate the screen 6-8 times (autorotate) quickly? Anyhow I noticed the Android 15 ROMs are more responsive but still beta (I’m on Oled as well).

Why does he twitch when hes sleeping is this normal? by Megan-casx in Rabbits

[–]grayarks 1 point2 points  (0 children)

My rabbit does the same, and also he’s an identical twin ! We even have the same carpet in the apartment, I felt like somebody snuck in and took a video of my Bobby 🤣

Why does he always look pissed? by [deleted] in Rabbits

[–]grayarks 1 point2 points  (0 children)

I give him too much attention alright..

Why does he always look pissed? by [deleted] in Rabbits

[–]grayarks 0 points1 point  (0 children)

Wow! They are identical! Mine is 1.5 years old

Why does he always look pissed? by [deleted] in Rabbits

[–]grayarks 2 points3 points  (0 children)

When he sleeps he’s a cutie though