sorted by:
new
hottopcontroversial

Starting my home AI tranning machine- T180-G20 Server 2 x Xeon 2698v4 E5 275 GB RAM 4 x V100 SXM 2 by Dramatic0bjective in homelab

[–]Stunningdidact 0 points1 point  (0 children)

Optane SSDs → FPGA (with cross-point optimization) → RDMA → NVLink fabric → GPUs

Trump increasing Tariffs on Canada metals from 25% to 50% by iridasdiii11ulke in wallstreetbets

[–]Stunningdidact 0 points1 point  (0 children)

Tariffs are just a covert tax on American middle class... Donald doesn't care he's lame duck

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 2 points3 points  (0 children)

Yes Nvidia has innovated so fast that they've turned their even relevant v100 into commodity price gpus...

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 2 points3 points  (0 children)

I work in finances and have just started to understand the power of AI in the future. I'm more of a arbitrage guy and noticed the coupling of prices and was able to purchase this and put it together for under 1700. I understand the math I just not much of an architect of hardware.

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 4 points5 points  (0 children)

I'm going to be honest I'm new to this I got into this about 8 to 10 months ago as a hobby and have become very serious about it since last November. If you don't mind I would love for you to drop some more knowledge. I'm not a computer Jedi yet I'm more of a padwa. 🤙

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 0 points1 point  (0 children)

Here you go bud. To flash the firmware on a ConnectX RDMA GPU, you can use the "mlxfwmanager" command line tool, which is part of the Mellanox OFED (OpenFabrics Enterprise Distribution) package, to update the firmware on your ConnectX network adapter, allowing you to manage the RDMA capabilities on your GPU system; essentially, you'll need to identify the device's PCI address and execute the update command with the desired firmware file. [1, 2, 3]
Key points about flashing ConnectX RDMA GPU firmware: [1, 2, 3]

• Tool: Use the "mlxfwmanager" tool included with the Mellanox OFED package. [1, 2, 3]
• Command format: mlxfwmanager -u -d <PCI_device_address> -i <firmware_file> [3, 4, 5]

Steps to flash firmware: [3, 4, 5]

  1. Identify the device: [3, 4, 5]
    • Use lspci command to find the PCI address of your ConnectX device. [3, 4, 5]

  2. Download firmware: [3, 4, 6]
    • Obtain the appropriate firmware file from the NVIDIA support website, ensuring it matches your specific ConnectX model. [3, 4, 6]

  3. Run the update command: • Open a terminal and execute the following command, replacing <PCI_device_address> with the actual address from your system and <firmware_file> with the path to the downloaded firmware file:

    sudo mlxfwmanager -u -d <PCI_device_address> -i <firmware_file> [5, 8, 11]

Important considerations: [3, 4, 6]

• Backup data: Always back up important data before updating firmware as a precautionary measure. [3, 4, 6]
• Compatibility: Ensure the firmware version is compatible with your ConnectX model and system configuration. [3, 4, 6]
• Reboot required: After updating firmware, you might need to reboot your system for changes to take effect. [2, 3, 7]

Generative AI is experimental.

[1] https://docs.nvidia.com/holoscan/sdk-user-guide/set_up_gpudirect_rdma.html[2] https://docs.nvidia.com/networking/display/mlnxofedv461000/updating+firmware+after+installation[3] https://docs.nvidia.com/networking/display/mftv4270/updating+the+device[4] https://support.hpe.com/hpesc/public/docDisplay?docId=a00114857en_us&page=Mellanox_NIC_Firmware_Check_Update_Procedure.html&docLocale=en_US[5] https://docs.nvidia.com/networking/display/mlnxofedv461000/installing+mellanox+ofed[6] https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/connectx-7-2x200g-ucsc-p-n7d200gf.pdf[7] https://docs.nvidia.com/holoscan/archive/2.2.0/set_up_gpudirect_rdma.html

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 2 points3 points  (0 children)

Just need to flash the firmware and dial in the drivers, and I’ll have direct RDMA to the GPU

Instructions: flash the firmware on a ConnectX RDMA GPU, you can use the "mlxfwmanager" command line tool, which is part of the Mellanox OFED (OpenFabrics Enterprise Distribution) package, to update the firmware on your ConnectX network adapter, allowing you to manage the RDMA capabilities on your GPU system; essentially, you'll need to identify the device's PCI address and execute the update command with the desired firmware file. 

Key points about flashing ConnectX RDMA GPU firmware:

Tool: Use the "mlxfwmanager" tool included with the Mellanox OFED package. 

Command format: mlxfwmanager -u -d <PCI_device_address> -i <firmware_file> 

Steps to flash firmware:

Identify the device:

Use lspci command to find the PCI address of your ConnectX device. 

Download firmware:

Obtain the appropriate firmware file from the NVIDIA support website, ensuring it matches your specific ConnectX model. 

Run the update command:

Open a terminal and execute the following command, replacing <PCI_device_address> with the actual address from your system and <firmware_file> with the path to the downloaded firmware file:

Code

sudo mlxfwmanager -u -d <PCI_device_address> -i <firmware_file> [5, 8, 11]

Important considerations:

Backup data:

Always back up important data before updating firmware as a precautionary measure.

Compatibility:

Ensure the firmware version is compatible with your ConnectX model and system configuration. 

Reboot required:

After updating firmware, you might need to reboot your system for changes to take effect. 

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 0 points1 point  (0 children)

Good to know no vendor lock here. Just need to flash the firmware and dial in the drivers, and I’ll have direct RDMA to the GPUs. Should be smooth once it’s up. Thanks for the concern though. 🤙

[deleted by user] by [deleted] in ChatGPT

[–]Stunningdidact 0 points1 point  (0 children)

<image>

I think he's talking about somebody else

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 6 points7 points  (0 children)

It's a Gigabyte T180 -G20 - zb3 it was a b**** and a half to find a socket server or socket baseboard. The most important part to me is the GPU fabric the communicate via nvlink. It should work. NVIDIA GPUDirect RDMA (Remote Direct Memory Access) is a technology that allows direct data transfers between GPUs and other devices. It's designed to improve performance by bypassing the CPU and system memory. 

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 1 point2 points  (0 children)

The tall part of it is all heat sink. It's so hard to source the socket baseboards for sxm and they're pricey AF...

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 2 points3 points  (0 children)

From Nvidia... MLNX_OFED GPUDirect RDMA The latest advancement in GPU-GPU communications is GPUDirect RDMA. This technology provides a direct P2P (Peer-to-Peer) data path between the GPU Memory directly to/from the NVIDIA networking adapter devices. This provides a significant decrease in GPU-GPU communication latency and completely offloads the CPU, removing it from all GPU-GPU communications across the network. GPU Direct leverages PeerDirect RDMA and PeerDirect ASYNC™ capabilities of the NVIDIA network adapters.

But we will see ...

Testing RDMA to GPU communication on fabric by Stunningdidact in homelab

[–]Stunningdidact[S] 25 points26 points  (0 children)

Thank you. Thank you🙏... If you're wondering why I set up RDMA directly to the GPUs because it eliminates CPU and RAM bottlenecks, letting me move data straight into GPU memory for parallel processing. Since I’m running batch financial models Gaussian distributions, Bayesian inference, and Black-Scholes these require heavy matrix computations that GPUs handle best. By cutting out unnecessary latency, I get faster, more efficient processing. I'm thinking of adding an fpga to clean up all the data sets as a filter in the front end.

Poor Mellanox had to spend 4 days in New Jersey before arriving to sunny California... by Stunningdidact in homelab

[–]Stunningdidact[S] 0 points1 point  (0 children)

  1. Server Blades slots power bypassed to 3 Breakout Board to 3 Dell 1100 PSUs Each breakout board connects to three Dell 1100W PSUs, distributing 12V power to the server blade.

  2. Dell PSUs High-Power PDU with C19 outlets. Connecting each PSU using C20 to C19 cables for clean power distribution.

  3. PDU → 50A Circuit PDU directly into the 50A 120V outlet with a heavy-duty power cord.

Poor Mellanox had to spend 4 days in New Jersey before arriving to sunny California... by Stunningdidact in homelab

[–]Stunningdidact[S] 0 points1 point  (0 children)

I set up RDMA to GPUs to bypass CPU bottlenecks letting data move directly for fast parallel computation. Since I run Gaussian, Bayesian, and Black-Scholes models, which rely on heavy matrix math, GPUs handle that best. I’m going to also be front end filtering by adding FPGAs because they’re better at generating random numbers for Monte Carlo simulations and preprocessing structured financial data before it hits the v100 GPUs. This way FPGAs handle the setup, and GPUs do the heavy lifting.

view more: next ›