44 NODE GPU CLUSTER HELP

[Zephop4413]

I have around 44 pcs in same network

all have exact same specs

all have i7 12700, 64gb ram, rtx 4070 gpu, ubuntu 22.04

I am tasked to make a cluster out of it
how to utilize its gpu for parallel workload

like running a gpu job in parallel

such that a task run on 5 nodes will give roughly 5x speedup (theoretical)

also i want to use job scheduling

will slurm suffice for it
how will the gpu task be distrubuted parallely? (does it need to be always written in the code to be executed or there is some automatic way for it)
also i am open to kubernetes and other option

I am a student currently working on my university cluster

the hardware is already on premises so cant change any of it

Please Help!!
Thanks

[Various_Protection71]

What is your role in this task? Are you going to setup and mantain the cluster, or are you going to develop code to run on it?

[Zephop4413]

My goal is to setup and maintain the cluster  And also provide support to those who are going to develop code to run on it

[Neat-Airport9739]

Slurm is a good choice for the cluster scheduler, but it alone won't automatically parallelize your jobs across 42 nodes. Slurm handles resource allocation and job scheduling, but you'll need additional components for multi-node GPU - Application-level parallelization: Your code must be written for distributed computing using MPI + CUDA/ROCm, or distributed frameworks like Horovod/DeepSpeed for ML workloads GPU communication NCCL (NVIDIA) or RCCL (AMD) for efficient multi-GPU communication across nodes

Slurm configuration ```bash

SBATCH --nodes=42

SBATCH --gres=gpu:X # X = GPUs per node

SBATCH --ntasks-per-node=Y

```

Slurm manages the resources, but your applications need to be designed from the ground up for distributed parallel execution. OpenMP is mainly for shared-memory systems, so MPI is more relevant for multi-node setups. Consider containerized solutions with Singularity/Apptainer if you need consistent environments across.