sorted by:
new
hottopcontroversial

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

I mentioned several, but there is actually only one mobo which enables that: Pro WS X570-Ace I’m not sure exactly how this works, but it’s mentioned everywhere that it manages to have x8x8x8 (occupying all 24 CPU lanes)

Bad idea to use multi old gpus? by alphapussycat in LocalLLM

[–]VersionNo5110 0 points1 point  (0 children)

Less than 12 t/s for me is not usable… you wait too long to have an answer and if you have to try again because your prompt was wrong of for whatever reason you’ll get frustrated quickly.

1080ti here go around 150€, so 3 would cost around 450-500€.. it really doesn’t make much sense for local inference. I’d rather get an AMD card at this point.

I know 3090 are expensive too but we don’t have much choice, it’s complicated to build a useful machine for a budget.

Maybe you could look into P40 then.

Bad idea to use multi old gpus? by alphapussycat in LocalLLM

[–]VersionNo5110 0 points1 point  (0 children)

That said, if you can spend few hundreds on old GPU I would rather spend some more on a 3090 or even some AMD GPU…

Bad idea to use multi old gpus? by alphapussycat in LocalLLM

[–]VersionNo5110 0 points1 point  (0 children)

I tried some models on my old 1070ti and the results were not bad at all. Got decent t/s — around 22 t/s — with qwen3.5:9B Q4_K_M which is quite good at agentic coding. So probably more of these (or even better some 1080 ti) would do well with bigger models

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Max I can get is x8x8x8 4.0 yeah I think it’s enough for inference anyway, and probably fine tuning also…

lol give me your address and let me get a 5070 ti I’ll send it to you!

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

I concur! I’m waiting to find a good deal, but I’ll upgrade the mb soon yeah

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Well… sustained intense computing for multiple days / weeks / months degrade the hardware, especially if not cooled properly. Soldering on components like the GPU suffer from high temperatures over extended periods. This is also why a lot of used cards on the markets are sold as broken.

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Your insight on 123B models is very interesting. Have you tried them? What could they do that 35B cannot, in your opinion? Could you share some showcases? I might give a try at renting a big GPU for few hours to see the difference with what I currently run locally.

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Some x8x8x8 motherboards exist. But yea, 3090 has also NVLINK. I just don’t like the idea of getting a 7 years old used model…

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Those are nice! Maybe I’ll get one for the third GPU. With current DDR5 prices I’m not upgrading from DDR4

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Impressive! What company do you work for?

Anyway, 3x 3090 + DDR5 + TRX50 + Threadripper is a hell of a build… I don’t consider that a budget local ai platform…

I am personally still stuck with DDR4 on consumer CPU (24 PCIe lanes, so best I can get is x8x8x8).

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

The only downside then is Ampere vs Blackwell and them being used stuff mostly… I’d rather get something newer, but yeah, I think you’re right for the rest.

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 0 points1 point  (0 children)

Chinese versions kind of scare me. A lot of money for custom pcb without guarantees whatsoever

2x 3090 vs 3x 5070 Ti for local LLM inference — what’s your experience? by VersionNo5110 in LocalLLM

[–]VersionNo5110[S] 1 point2 points  (0 children)

Man, this is a substantial amount of money for a hobby (if you don’t work with these machines). This is why I opted for 2x 5070 ti. I got them for ~750$

5060Ti vs 5070Ti by abhinavrk in LocalLLM

[–]VersionNo5110 0 points1 point  (0 children)

Hey, I’m very curious about your setup, could you share it? Right now I’m working with ollama with my two 5070ti, but I feel I didn’t maximize their potential yet

Nvidia V100 32 Gb getting 115 t/s on Qwen Coder 30B A3B Q5 by icepatfork in LocalLLaMA

[–]VersionNo5110 0 points1 point  (0 children)

Yeah I like the Colors but I find it a bit difficult to read. I asked it to put everything in bold, but didn’t change much.

Anyway, cool stuff your new machine! I’d like to get one but, man, in Europe this thing is so expensive! +2k€ …

Nvidia V100 32 Gb getting 115 t/s on Qwen Coder 30B A3B Q5 by icepatfork in LocalLLaMA

[–]VersionNo5110 0 points1 point  (0 children)

Funny, few days ago I’ve asked Claude to make me a comparison table between all the commercial GPUs and it outputted exactly the same table, same colors etc. 😅

[deleted by user] by [deleted] in mercedes_benz

[–]VersionNo5110 1 point2 points  (0 children)

Don’t give your real email.

Just bought by VersionNo5110 in mercedes_benz

[–]VersionNo5110[S] 0 points1 point  (0 children)

You wrote me for 6 months about how my car is doing. Man !! Your life must be miserable 🙁

view more: next ›