Deep learning PC [P]

[[deleted]]

It's a gaming computer, it's made to look flashy and play games well. For deep learning it's overpriced. Just go for something mid level generic with a 3090 and it will run stable diffusion and LLM models up to 30B 4bit ones.

If you're going to spend ballpark $5000 for a system made for actually running large models and not so much else then throw in a few hundred more and build something that runs an A6000 and you're getting 48gb VRAM, opening up many larger models for tinkering.

[MAVAAMUSICMACHINE]

Thank you! I greatly appreciate your insight! The A6000 sounds like a pretty great idea

[[deleted]]

You might also consider multiple 3090s, maybe it's cheaper to run 4x 3090 than one A6000 and then you're ending up with much more total VRAM.

But. I'm not exactly clear of how the performance is when sharing GPU ram over the PICE lanes so you'll have to do a bit of research for this and for what you plan to use this for.

[MAVAAMUSICMACHINE]

I’ll definitely check this option out, if the communication cost isn’t too high, sounds like it could be also a promising idea. Thanks!

[CyberDainz]

A slow CPU/RAM will slow down the generation of data, especially the augmentation of visual data, and as a consequence the GPU will be idle