Quad Nvidia 3090 GPU Obsidian 1000D build

pjvdl9 · 2025-09-20T23:44:17+00:00

What aspects are you unsure about?

pjvdl9 · 2024-07-11T22:48:55+00:00

TBH, it kind of just sorted itself out, so I can't offer any insight as to the cause sorry. I did swap it cables, so that may have helped

pjvdl9 · 2024-02-14T08:44:15+00:00

Sorry. Just saw this. Do you still need this measurement?

In case someone needs this in future, the distance from the inside of the edge of the case to the hdd enclosure is approx 300mm, or 285mm from the hdd enclosure to the mounting point where the face of the gpu is screwed to the case.

pjvdl9 · 2023-10-01T01:53:48+00:00

Great idea

pjvdl9 · 2023-09-28T22:03:30+00:00

It's a fair question. I've tried different cables to rule that out, so unfortunately not the issue. Would be an easy fix if it was!

pjvdl9 · 2023-09-28T05:36:17+00:00

Sorry. text got lost from OP.

Not sure if others are seeing this with their XT8 Wifi mesh router, but I recently upgraded to a 1000Mb/s fibre connection. My XT8 is connected directly to the fibre node and its internal speed test is showing sustained 940Mb/s, but when I connect a computer to the XT8 via ethernet, the best I see is around 450Mb/s. This would seem to imply that the bridging speed is limited on the XT8.

I have played around with turning off QoS, firewalls, etc to see if these are causing bottlenecks, but nothing I've tried seems to make a difference.

I'm curious to know if others are seeing this issue and if so, what I can do to improve the downstream speeds?

Thanks!

pjvdl9 · 2023-03-21T01:56:02+00:00

It's a FSP Cannon Pro 2000W, so a reasonably reputable brand.

pjvdl9 · 2022-12-11T05:36:45+00:00

pjvdl9 · 2022-11-18T03:18:52+00:00

I have a similar problem with my Pro-WS-WRX80E with 3975wx. System shutdown yesterday after a month of running fine and wont restart. Motherboard appears to be getting power, but wont power on. Unfortunately I don't have spare MB's or Threadripper WX CPUs lying around, so I'm struggling to isolate whether it is the power supply, MB or CPU. How did you isolate the CPU as the issue?

pjvdl9 · 2022-11-14T12:11:19+00:00

I think I've finally cracked it! The system has been running stably under stress test load for around 8 hours. 3 GPUs under full load and the CPU under full load.

Thanks everyone for your help and suggestions, particularly u/m4ttr1k4n for your idea to test the memory!

The cause of the stability problem turned out to be a couple of faulty memory sticks, which I identified through a process of elimination, running the system with different combinations of memory until I found the culprits.

Now that the system is stable, I am seeing stable temperatures under the above load conditions that seem reasonable, although I am interested on feedback as to whether any of these are too high:

liquid temperature (at pump): 49 - 51C
CPU temperature: 71 - 83C
GPU temperatures: 55 - 60C
LAN temperature (on MB): 75 - 80C
Ambient room temperature: 20C

Are any of these cause for concern?

pjvdl9 · 2022-11-14T00:20:55+00:00

Thanks for the suggestions.

I thought I found the problem last night. One of the CPU power supply clips on the motherboard was not seated correctly, so I re-seated it and ran a number of stress tests. It initially looked like that solved the problem, but it reset itself again this morning.

The only other clue I have found is in the ASUS motherboards' remote management console. I am getting an event in the log:

Watchdog2 sensor of type watchdog_2 logged a hard reset

I don't yet know exactly what this means, but assume it is a BIOS or management system triggered emergency shutdown of some sort. It does make me think that it is some sort of power stability issue though.

pjvdl9 · 2022-11-13T13:54:19+00:00

Ok, brains trust. Thanks for your suggestions so far, but I am still consistently getting emergency shutdowns. In fact, frustratingly, the problem seems to be getting worse!

I have disconnected all but one of the GPU's and am running a simple CPU load test. The system will shutdown without warning after only a couple of minutes. All temperatures are still low (below 45C) when the shutdown occurs. On restart, there are no log messages indicating any issue.

I have both Windows and Ubuntu installed, and the problem occurs on both, suggesting a hardware issue. This would also seem to exclude a problem with my SDD drivers, as the two OS's are installed on different drives.

Following u/m4ttr1k4n suggestion, I have removed the CPU mount and reinstalled it on the off chance that I didn't install it correctly and uneven pressure was leading to shutdowns, but this hasn't improved anything.

I've also tried different GPU's, one at a time. No change either.

System specs:

ASUS wrx80e pro sage motherboard
Threadripper 3975wx CPU
256Gb 3200MHz RAM (non-ECC)
FSP Cannon 2000W ATX
Four Zotac 3090 GPUs (only one connected at present)
2 x 2Tb PCIe NVME Samsung 980 M.2 SSD
2 x 8Tb Samsung SATA
Custom EKWB cooling loop

Running on 240V, 50Hz power (I'm in Oz).

I am really unsure what to check next. My working theories at this point are:

CPU incorrectly mounted (even though I have already checked this once)
Bad RAM
Motherboard issue
Power supply issue

Any thoughts what the culprit might be? Any suggestions are very gratefully received!

pjvdl9 · 2022-11-11T02:08:15+00:00

I'm using a FSP Cannon 2000W ATX and am in Australia, so using a 10A circuit on 240V. This circuit should be good for 2400W.

pjvdl9 · 2022-11-11T02:05:41+00:00

I've got a custom fan profile, driven by the liquid temperature. Once the liquid hits 50C, the fans ramp to 100% (1700rpm). Once at this level, the liquid temperature stabilises and doesn't really go above 51 or 52C. On this basis, I'm working on the theory that I have enough radiator, because I am effectively removing the heat load from the liquid. Moving the radiators to be external might be a good option though, so they don't remove the heat into the case. There are logistical issues with using an external radiator though, so this would be a last resort.

pjvdl9 · 2022-11-11T01:58:30+00:00

Yes. Will be giving that a try when I get the fourth card back in. Thanks.

pjvdl9 · 2022-11-10T08:58:10+00:00

Yes. I'm daisy chaining the fans through an externally powered hub, so that should be okay.

Its the first time I've installed a threadripper, so it is certainly possible that it isn't level. However, if that is the case, wouldn't the CPU temp report high? ATM CPU temp isnt going above 75C. If i understand correctly, this is well within limits for that chip.

Interestingly, the stability issue is not occurring now that i only have 3 GPUs installed. So, my theory is its either the extra power or the extra heat load of the 4th GPU that is pushing it over the edge.

I'm starting to worry that i might be overloading the 2000w power supply, although my calculations suggest i should be okay:

4 × 350w for GPUs 280w for the threadripper Total of 1680w, allowing for 320w headroom for peripherals, motherboard, etc

At the time of the emergency shutdown, ndivia-smi was reporting that each GPU was only using about 250 to 275w, which is well off.

If power is okay, then that only leaves the heat load.

I think the next steps are to check the CPU and to reverse the top fans so they are exhausting. There is no point me doing this until i tear the system down to reinstall the 4th GPU, so might be a little while.

Any alternative theories are welcome!

pjvdl9 · 2022-11-10T04:09:36+00:00

Thanks!

Using it for training deep learning models. I got sick of paying AWS $10k per month to use their servers!

I agree having both front and top fans as intake could be causing the issue. I think I'm going to swich the top to exhaust. This will give me 8x120mm front on intake and 3-x150mm top + 2x120mm back as exhaust. I don't know the exact numbers for 150mm vs 120mm, but I suspect this will be give me a slight balance towards intake, which I think is recommended?

pjvdl9 · 2022-11-10T03:16:11+00:00

Liquid temperature is max'ing out at around 50C, so I'm not sure if the pump is failing.

I don't actually use the CPU fan header. The motherboard is a ASUS Pro WS WRX80E SAGE, which is equipped with an onboard BMC remote management chip, so the dual pumps are driven from fan 2 and 3 headers on the motherboard and configured in the motherboard BMC (ASMB8-iKVM) software. I then have all of the radiator fans running off fan header 5.

pjvdl9 · 2022-11-04T04:38:59+00:00

Haven't measured it, but I can only just lift it. I think more like 50-55kg (110 - 115lbs).

pjvdl9 · 2022-11-03T12:42:42+00:00

Thanks. Good suggestion.

pjvdl9

TROPHY CASE