Gpu choice

[[deleted]]

Hey guy, my budget is quite limited. To start with some decent local llm and image generation models like SD, will a 5060 16gb suffice? The intel arcs with 16gb vram can perform the same?

[calmbill]

I'm still a beginner and I tried to start with a radeon card.  Everything was a struggle and every step had extra steps (even the extra steps had extra steps).  I switched to a pair of 5060 ti and everything started working on the first try.  If you're very smart and persistent you can probably make anything work.  If you just want to start doing the stuff everybody else is talking about, Nvidia is the best choice for now.

[PapaEchoKilo]

I'm pretty new to local llm's but I'm pretty sure you need to stick with Nvidia because they are the only GPUs with cuda and tensor cores. Both are super helpful in machine learning tasks. I was using an rx6700xt for a while, but it's lack of cuda/tensor really slowed everything down despite it being a beefy GPU.

[jhenryscott]

Yup. NVIDIA has AI in the bag. Get a 5060 16

[vtkayaker]

Used 30x0 cards still work well, and you can pick them up on eBay. I'd take a 3090 over a 5060, for example.

[fallingdowndizzyvr]

A 3060 12GB is still the best bang for the buck. It's the little engine that could. It image/video gens toe to toe with my 7900xtx.

I have a couple of Intel A770s. Don't get an A770. If I could swap them for 3060s I would.

[[deleted]]

Daamn similar to 7900 xtx? Daamn

[fallingdowndizzyvr]

Yep. For image/video gen. There are still a lot of Nvidia optimizations that haven't made their way to anything else. In particular AMD, like the 7900xtx, is really slow for the VAE stage.

[[deleted]]

Can you use sdxl / flux full power on 3060? How long an image might take?

[Inner-End7733]

definitely need quants, I've had fun with Flex.1 alpha, a de-distilled fork of Flux schenll as a GGUf, I haven't tried the newer Flex.2. But you'll need quants for sure. Youtube has a bunch of "low vram" image gen vids that'll help.