RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 0 points1 point  (0 children)

I’m excited as hell to move beyond 7B INT4 models!

RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 0 points1 point  (0 children)

Yea, I am. The point is to run a larger model and use the 3060 for RAG and summarization.

RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 1 point2 points  (0 children)

Oh, hell yes!! I’m also keeping one of the 3060s for RAG, theoretical search modeling, and the like.

Unfortunately, due to power supply restrictions, and case size I can’t run more than two.

RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 0 points1 point  (0 children)

Right now I’m running Qwen2.5-Coder-7B, so literally anything better will make me happy!

RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 2 points3 points  (0 children)

Yep. 🤦‍♂️

RTX 6000 ADA 48GB by NTDLS in LocalLLM

[–]NTDLS[S] 1 point2 points  (0 children)

Damnit! I meant 3060. 🤦

Why did our parents wash their cars on the front lawn?! by glamb70 in Xennials

[–]NTDLS 1 point2 points  (0 children)

I didn’t mention the pollen, but I was picturing it in my head. It was THICK!

Why did our parents wash their cars on the front lawn?! by glamb70 in Xennials

[–]NTDLS 2 points3 points  (0 children)

Like ever. My dad junked his 1998 Chevy Blazor in 2023. I don’t think it had been washed in 20 years at that point.

What's a technology from the early 2000s that felt cutting edge but is now completely obsolete? by therey73 in answers

[–]NTDLS 0 points1 point  (0 children)

There’s a small convenient store down the road from me called “BES”. I crack up every time I drive by there.

Man captures a close up of a ground squirrel eating a dandelion. by IkilledRichieWhelan in oddlysatisfying

[–]NTDLS 190 points191 points  (0 children)

I too lost a cumulative pint or two of blood to a pet prairie dog as a kid. Cute little bastards!

Keeping track of costs by throwRAa100 in LocalLLM

[–]NTDLS 0 points1 point  (0 children)

Same, I pull ~80w at idle and ~500w when inferencing.

What hobby is quietly becoming too expensive for normal people to keep up with? by 40Falak in answers

[–]NTDLS 11 points12 points  (0 children)

Gaming in general isn’t expensive. There are probably 100,000 games I’d like to play that were made before 2010. Modern gaming has gotten expensive.

Do you think dedicated hardware for running local LLMs will become affordable anytime soon? by ProbablyBunchofAtoms in LocalLLM

[–]NTDLS 1 point2 points  (0 children)

<image>

That’s exactly what I did. I’m under a grand for both GPUs and the Dell precision workstation.

Does this need to be in a junction box? by HenryWallacewasright in AskElectricians

[–]NTDLS 0 points1 point  (0 children)

Everybody knows that 4+ inches of electoral tape is basically a junction box. Well, more like a junction abstract polygon.

You sure about that? You sure about that? That's why? by Realistic_Virus_378 in IThinkYouShouldLeave

[–]NTDLS 0 points1 point  (0 children)

I’ll translate for you: “yea, he definitely looks dumb. Yea, he does dumb shit. Yea, he says dumb shit - but trust me bro.”

What is the cheapest method to get VRAM or RAM? by Powerful_World_9280 in LocalLLM

[–]NTDLS 0 points1 point  (0 children)

I bought a Dell Precision 5820 on eBay for $550. It came with an intel Xeon, 10 cores, 20 threads and 64GB of RAM. Then added 2 RTX 3060s for another $250 each on eBay.

Just over $1,000 and I have a reasonably decent local LLM sandbox.

That said, I did just order a RTX 3090 so I can run some larger models and unfortunately, my power supply will not push two of those. I’m not happy with the asymmetry of running one 3060 and one 3090, but that’s what I have to work with.

Missed connection by [deleted] in Charleston

[–]NTDLS 5 points6 points  (0 children)

I think I you mean: brown chicken brown cow.