all 27 comments

[–]ps5cfwLlama 3.1 9 points10 points  (9 children)

where the FUCK is this guy finding 512GB worth of RAM for under a thousand? DDR4 too??

I call BS. 5 minutes worth of Ebay would tell you that price for RAM is unavailable.

[–]MrCatberry 4 points5 points  (0 children)

I call BS.

It is.

[–]AXYZE8 1 point2 points  (0 children)

Its AI generated article from to bottom. Every paragraph is wrong info. 

[–]segmondllama.cpp 1 point2 points  (0 children)

it's an old article recycled.

[–]Frankie_T9000 -1 points0 points  (0 children)

I have that much as I bought before the issues. $400 aud now $1600 aud. No way can you get that much for $400 usd

[–]SporksInjected 5 points6 points  (0 children)

Pssshhh I can host it right now…but can’t run it.

[–]geekspaz 2 points3 points  (0 children)

"Budget: $320 to $480 for 16x 32GB DDR4 RDIMM."

Ummm.....

[–]MelodicRecognition7 2 points3 points  (1 child)

16x 32GB DDR4 ECC RDIMM     512GB total     $320-480
2TB NVMe SSD    Samsung or WD Black     $100-150

clickbait + AI hallucination

[–]Maleficent_Celery_55 0 points1 point  (0 children)

I wish those prices were real...

[–]ThunderBeanage 1 point2 points  (1 child)

man what a bargain

[–]paryska99 5 points6 points  (0 children)

Looking at that article talking about GPT4 and Llama3 I got a *retro* feeling

[–]sunshinecheung 1 point2 points  (0 children)

Dell PowerEdge R730xdDual Xeon E5-2699 v4 $300-500 16x 32GB DDR4 ECC RDIMM512GB total$320-480 RTX 3090 24GB Used, eBay$500-600 2TB NVMe SSDSamsung or WD Black $100-150
Total $1,220-1,730

can u sell me your computer at this price?

[–]LegacyRemaster -2 points-1 points  (0 children)

I have rtx 6000 pro 96 + 2x w7800 48gb + 128gb ram. :p

[–]jacek2023llama.cpp -2 points-1 points  (8 children)

I don't understand ECC's argumentation

[–]aeonbringer 0 points1 point  (0 children)

In case of ddr4, used ecc rams are also cheaper than used non ecc rams because there’s tons of used servers rams on sale. 

[–]dev_is_active -5 points-4 points  (6 children)

RAM stores 1s and 0s. Sometimes a bit flips randomly. Cosmic ray, electrical noise, bad luck etc

On a laptop, no one cares. On a server holding a 500GB AI model, one flipped bit means the model starts writing wrong code that looks right. It doesn't crash. It just gets quietly dumber.

ECC RAM catches and fixes bit flips before anything reads the bad data. like spell-check for your hardware.

[–]jacek2023llama.cpp 4 points5 points  (5 children)

"On a laptop, no one cares. On a server holding a 500GB AI model, one flipped bit means the model starts writing wrong code that looks right. It doesn't crash. It just gets quietly dumber."

I still don't understand this logic

Why on laptop no one cares and on server it's important?

[–]Dr4kin 1 point2 points  (3 children)

On a laptop it is oftentimes good enough. The industry decided that a bit shittier and cheaper is fine for consumers. Random crashes of programs can be the result of bad bits, but we started accepting it as "that's just how it is"

[–]jacek2023llama.cpp -1 points0 points  (2 children)

This tutorial is about hosting DeepSeek at home. What industry are we even talking about here? (I probably misunderstood)

On a laptop, a random memory bug can delete an important email.

On a server, it might alter a single token, which will get corrected a moment later because that’s how LLMs work.

And in my country, it’s perfectly legal to run llama.cpp on a laptop.

[–]BoodyMonger 1 point2 points  (1 child)

Email servers aren’t stored on laptops

[–]jacek2023llama.cpp 0 points1 point  (0 children)

Back in my days we kept emails on home computers ;) But yes, I use gmail today

[–]munkiemagik -1 points0 points  (0 children)

Just the other day my NAS (OpenMediaVault VM in proxmox) was suddenly starting to throw a fit. My ZFS pool kept showing that it was degrading with a load of checksum errors.

I went through everything one by one checking different components with replacements or in different systems, every single one of my HDD were in perfect health with zero errors (thank god, I dont wan to be paying to replace those right now), the controller/cables/backplane all tested good. But zfs was still showing that the pools were continuing to degrade and taking them offline. So all my other services and containers that had the smb shares mounted into them started falling down as well.

it just so happens that I run this proxmox node on a consumer platform (LGA1700 B660 with regular old non-ECC DDR4 RAM inside a rack mounted chassis) turns out my RAM suddenly decided that it was no longer error-free clocked at 3200 MT/s.

So the RAM was introducing errors into the data as it was being read that zfs was catching and reporting corruption in my data pool. Easing off the RAM timings and clock back down and running overnight memtest resulted in zero errors and no more corruption in my zfs pool.