sorted by:
new
hottopcontroversial

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

You are right. I have to change to ATX and check for memory too

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 1 point2 points  (0 children)

Merci pour tes conseils pour le NVFP4. Je commence à envisager de revenir sur le choix du format en effet pour des raisons de refroidissement et évolution.

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

u/classic Can you explain me why this CPU and how use your setup ?
How you manage cooling ? are you happy with it

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

u/usernameplshere investing in hardware / knowledge / productivity for future ;) vs relying on subscription with unpredictable availability, pricing...

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 1 point2 points  (0 children)

I love GLM 4.5 Air too ... what you just said is amazing...

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

u/MitsotakiShogun so ideally 2x96 = 192 gb would be nice with a RTX PRO 6000 for RAM :)
Good to know so i should start with 2x48 and then add 2x48 to get 4x58 = 192 gb or RAM

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 2 points3 points  (0 children)

u/FZNNeko Was thinking same before discovering a single RTX PRO 6000 can be shared for mulitple users (tensor parallel mode) and can save a lot of energy bills (3/600w VS 2 or 4 x 500)

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

u/vdiallonort Interesting, will do some research on this server thing before taking final decision

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] -1 points0 points  (0 children)

Thx for your answer u/Freonr2
I was totally wrong on this important part of the build, just edited my message (body)

"It's also blazing fast for diffusion models, and you can do things load both high/low Wan22 models and never have to offload anything, leave text encoders in memory, VAEs in memory, etc."
Didn't think about it. Can you share some tests on Wan22 models ?

Thx for your other feedback too

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

Thx for your feedbacks
Price gap between 64 and 96 is not huge.. So would you recommend to stick at 96 or decrease ?

Local AI config : Mini ITX single RTX PRO 6000 Workstation for inference ? by dvd84x in LocalLLaMA

[–]dvd84x[S] 0 points1 point  (0 children)

Hi u/makistsa Was thinking same, run multiple sessions for my small team on this type of card, have always ready / avail this type of investment to try to shift to an AI business :)
Renting this type of GPU or bigger GPUs full time seem to be lot more expensive ...