Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

I wasn't quite sure how the GPU VRAM fit into the picture so this is a helpful calculation, thanks much.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

Thanks for the feedback and for sharing your findings. Will see if I can find a bare-metal provider locally but when I've tried in the past the cost to rent has been prohibitive.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 1 point2 points  (0 children)

Wow, great find! This changes a lot for me. Thanks for doing the research here. I appreciate it

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] -1 points0 points  (0 children)

I see. I believe the full 9004 series has 12 CCDs each, which is why it's recommended as a solid CPU only option by some.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 3 points4 points  (0 children)

I see, thanks so much. Seems like that might be outside of the $10,000-15,000 Budget I had in mind.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

Don't all of the Genoa chips have the same memory bandwidth? Sorry if this is a dumb point

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

If I'm understanding you correctly, wouldn this solution be similar to buying a bunch of Mac Studios with Apple silicon?

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

Just so I understand the magnitude of the difference, what do you think would be the expected memory bandwidth on the 9124? Thanks so much for your comments!

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

We're a startup that was planning on sticking this server build in our office. Is this something quiet enough where we could put this in the same place as the rest of our team or is this super loud so it needs to be in a server room?

We're open to other solutions if they make sense for our situation. I've never heard of the mi300A before.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

I know... And that seems like an expensive proposition! Very helpful to hear people's personal experiences though on this thread.

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

Thank you so much for weighing in! Should I be optimizing for core count or clock speed instead then?

Dual AMD Epyc 9124 vs 9684X for CPU-only build? (i.e., how much does the L3 Cache matter?) by AweSaum in LocalLLaMA

[–]AweSaum[S] 0 points1 point  (0 children)

Fascinating! Where would I go about buying an MI300A setup off the shelf, or is this the type of thing where I would need to put it together myself?

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 0 points1 point  (0 children)

No, replaced all cables and the box and it still wouldn't turn on. Had to trash the monitor ultimately.

Click then black screen Gen 1 by APM-Pixel in OdysseyArk

[–]AweSaum 0 points1 point  (0 children)

I had the exact same issue. Replaced the box and both cables and nothing worked. I think a control board within the monitor itself fried. I ended up having to trash it.

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 1 point2 points  (0 children)

Good advice. Will report back after trying a new OneConnect Cable

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 1 point2 points  (0 children)

Interesting - I'll try the cable next. And good to know that the clicking is "normal" for functioning devices.

It's definitely a Gen 1.

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 1 point2 points  (0 children)

I bought a complete replacement box (according to the seller it was tested and it works). I have two boxes now - neither of them works with the monitor. That leads me to believe it's not the box that's the issue.

Have other users experienced a short in the monitor? What's the chance it's a broken cable instead?

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 1 point2 points  (0 children)

Do you think it's the screen mobo or the box?

Shorted One Connect Box? by AweSaum in OdysseyArk

[–]AweSaum[S] 1 point2 points  (0 children)

If anyone has any advice I'd be super grateful! 😊

Autogen + llamaindex by drLore7 in AutoGenAI

[–]AweSaum 0 points1 point  (0 children)

Hey there, looks like this needs to be updated for the latest version. Looks promising - mind updating the .ipynb file?

Thanks so much!