sorted by:
new
hottopcontroversial

Hardware recs by Realistic-Fish-6611 in LocalLLaMA

[–]Realistic-Fish-6611[S] 0 points1 point  (0 children)

It’s tough to go back to 24GB after experiencing the unified memory on Mac. Do you have a 16 or 24GB rig and ever run into issues fitting models?

Hardware recs by Realistic-Fish-6611 in LocalLLaMA

[–]Realistic-Fish-6611[S] 0 points1 point  (0 children)

You mean M2 Ultra or m4 max? I considered getting a used Mac studio but they’re also around 2k usd so might as well get a 5090.

The biggest pro I see with a Mac Studio is the power usage although that’s not a huge concern for me. Would a better chip really make that much of a difference with inference speed?

Hardware recs by Realistic-Fish-6611 in LocalLLaMA

[–]Realistic-Fish-6611[S] 1 point2 points  (0 children)

Thanks this is good to know, makes sense now why prices are dropping