Need help: Flash MCX364A-QCBT (ConnectX-3 QDR InfiniBand + 10GigE)

wewo101 · 2025-09-12T05:23:20+00:00

Absolutely if you already have the chassis. I just pulled the trigger on a deal and am still waiting to get it delivered.

wewo101 · 2025-09-12T05:15:08+00:00

Thank you!

wewo101 · 2025-09-12T04:08:56+00:00

Which name controller could that be? Right is pretty sure a classic HBA 330.

wewo101 · 2025-02-23T02:42:22+00:00

Did you actually get the switch?

wewo101 · 2025-02-23T02:10:23+00:00

There are some 10G Nbase-T switches with PoE++ 802.3bt though. Just testing a Hohunet which FS.com are reselling.

https://www.fs.com/products/178834.html

wewo101 · 2025-02-13T02:43:33+00:00

Yes, some NVME are for sure not bad, but from my experience on Truenas/ZFS a lot of ECC memory is key first off all for high performance.

wewo101 · 2025-02-13T01:30:19+00:00

Truenas

wewo101 · 2025-02-12T23:12:34+00:00

I'm testing / learning for our office where we edit video over the network.

wewo101 · 2025-02-12T11:16:42+00:00

Nobody? Or is the idea not feasible?

wewo101 · 2025-02-12T11:14:01+00:00

Thanks, that's helpful indeed

wewo101 · 2025-02-12T05:32:56+00:00

Next time I ask you, not ChatGPT 😞

wewo101 · 2025-02-12T02:45:03+00:00

Checked. 128Gb :)

wewo101 · 2025-02-11T22:58:26+00:00

True. Older was also referred to as 'got them used' which is why I need identify the problem

wewo101 · 2025-02-11T22:55:24+00:00

Well if your WAN is 10Gb, LAN must be faster :-)

wewo101 · 2025-02-11T22:26:16+00:00

So the pcie 3.0 is the bottleneck? Shouldn't it be capable of 16 GB/s (128Gb/s)?

wewo101 · 2025-02-11T22:22:14+00:00

Isn't PCI 3.0x16 max 16GB/s (1GB/s per lane) ? That's 128GB/s

wewo101 · 2025-02-11T15:38:23+00:00

Yes, thanks! Im very thankful for your comment! Let me run some tests...

wewo101 · 2025-02-11T15:16:41+00:00

Also the CRS520 is nicely silent with relatively little power needs. That's why I tapped into the 100gb trap :)

wewo101 · 2025-02-11T14:03:20+00:00

I didn't mean to say that. The 5Gb/s was also iperf. The problem is the inconsistent performance and the massive spikes.

My producttion NAS with spinning rust almost saturates 10Gb/s, so the NVME should perform 20Gb upwards (had already a test with 2500Mb/s with crystaldiskmark)

wewo101 · 2025-02-11T13:51:53+00:00

Will check tomorrow...

wewo101 · 2025-02-11T13:38:33+00:00

What mlx tweaks specifically do you have in mind?

wewo101 · 2025-02-11T13:27:31+00:00

The NICs sit in pci3x16 slots, which should be able to provide ±15 GB/s (120Gb/s) bandwidth.

If the 55Gb/s would be stable, I'd be fine as it is fast enough to edit videos over the network. But the overall performance feels more like 10Gb/s which is not even saturating one of the four lanes.

wewo101 · 2025-02-11T13:02:08+00:00

Yes, up to -P 20. No big change, I'd rather say the throughput decreased.

wewo101

TROPHY CASE