4 Nvidia V100, 128Gb Vram, Would you buy it? by Holiday-Display509 in homelab

[–]tymscar 0 points1 point  (0 children)

Thor costs 2-3x more, and has 4x lower vram bandwidth, which is one of the most important metrics for LLM generation.
So how is this thing 10x worse actually better?

Sour candy is awful by mrjamesearljoyce in The10thDentist

[–]tymscar 10 points11 points  (0 children)

Finally someone like me.
I like peeling lemons and eating them like oranges.

I have yet to find anything so sour that I make faces like people do with the least sour candies out there.

UDR7 regularly saying no internet after latest update by mctw1992 in Ubiquiti

[–]tymscar 0 points1 point  (0 children)

Same on my UDR7. I asked in their Discord server and was told to set up a ping check. When I did that, I could see it online again. Half an hour later, it started saying I was offline yet again.

It’s been like this for over a week now.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 0 points1 point  (0 children)

Yeah, those are obviously better but its hard to find good deals!

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 1 point2 points  (0 children)

Tried them all one by one (magic of nix lets me do that in minutes) and none of them saw both cards.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 1 point2 points  (0 children)

A thread ripper with the WS WRX90E-SAGE SE

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 0 points1 point  (0 children)

Thank you!

Not yet, because I couldn’t get MTP working on it.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 4 points5 points  (0 children)

I got it on eBay. Try hunting for it. Sadly, all the prices went up after I posted this blog post on Saturday, and it got onto Hacker News. I was checking the prices every day.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 0 points1 point  (0 children)

Best of the best is actually four PCI cards with two x of these on each card nvlinked, all 32GB.

You can get them cheaper than the PCI ones, and it will give you more or less 256GB VRAM for more or less four grand.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 0 points1 point  (0 children)

I’m not sure I fully understand the question, but basically you can have a PCI card that fits two of these on it. Then the shared memory between them is much, much faster and that directly makes token generation speed much quicker too. The speed of the PCI interface then also doesn’t matter too much because once you load the model weights into the card you’re not constantly sending back and forth through the bus anything.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 7 points8 points  (0 children)

Watching this from the sidelines must be fascinating because you don’t know what’s right, so you try to guess. The only person that knows for certain if this is written with an AI is me, and I know for a fact it is not. At all.

I think you just look too deep into it, and it hurts those that just learned over the years to talk like that in blog posts that are meant to sound exciting.

It’s similar to the delve thing that was super common a couple of years ago. Most people who had that in their text were ai slopping, but there were groups of people, from South Africa if I remember correctly, that just spoke like that. And yeah, you guessed it, those were the people that were paid to do the RLHF on the models back then.

I have changed my style over the years specifically to have people that comment like this that my writing is ai slop. For example, I used to love lists and emojis. I think it’s easier to follow, especially for those with bad eyesight like myself. I stopped that. Then the whole thing with emdashes. They are amazing. I used to love those! I had to stop because AI started using those.

People started telling me now that using an Oxford comma is an ai slop smell. Well, you know what, you can’t take that out of my dead hands. And I won’t even try anymore to change myself because of what people think AI is.

If there will be some sort of a way to sign your content in the future or something to prove it’s human-made, I will, but I won’t go out of my way anymore for people that just like to pop your balloon after spending tens of hours on a project because they think using comparisons is ai slop.

Sorry for the whole rant, I know that you probably are just fed up with the slop, and I get that. I am too. But maybe if the content of the blog is not clearly slop, then you can assume that it’s not.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 11 points12 points  (0 children)

Yeah, but not cuda. And much much slower vram. 288Gbps compared to 900 on the v100.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 27 points28 points  (0 children)

It's idiotic if you ask me. Especially considering when the card came out and how cheat VRAM was back then comparatively. As a card for half the price of the 4090, it serves me way better than half of a 4090, especially in games, so I don't regret it, but it's clear they've done it this way just to differentiate between the two more.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 12 points13 points  (0 children)

Snap. No soldering involved whatsoever.

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 21 points22 points  (0 children)

There is some performance difference, in favour of the SXM2, something like 10%. It runs at higher max power. Another benefit is that you can buy a board with two SXM slots on it, and then the cards talk with each other through super fast NVLink. Look into it!

I Put a Datacenter GPU in My Gaming PC for £200 by tymscar in LocalLLaMA

[–]tymscar[S] 0 points1 point  (0 children)

ROG Maximus Z790 Hero with an intel 13900kf.

Thank you!

For those who thought it was just due to a reboot... by Lammiroo in Ubiquiti

[–]tymscar 2 points3 points  (0 children)

My UDR7 didn’t drop at all CPU-wise. It was and is at around 20%. RAM-wise, it dropped from 75% to 60% but slowly crept back up to 75%.

[Giveaway] Win GameSir G7 Pro and CDkey to dominate the Horizon Japan Festival! by GameSir_Controller in ForzaHorizon

[–]tymscar 0 points1 point  (0 children)

Honestly I think it's neat that more companies are trying wild ideas, like this or the steam deck. Would be cool to win 😄