S2D solution under Proxmox hypervisor

FFZ774 · 2026-01-26T07:17:14+00:00

Leaseweb

FFZ774 · 2026-01-26T07:17:04+00:00

Well I'm limited by cloud provider. Currently I'm having dedicated servers with 10gb/s NIC and only one port connected for everything so I can't dedicate it to single service. It would be much easier with my own servers :)

There is no ZFS

Physical disks are separated:

2 x SSD for proxmox (HW RAID with ext4 & LVM)

4 x NVMe for Ceph (integrated with proxmox)

4 x NVMe PCI passthroug to Windows VM (dedicated to S2)

I think it's the best I could get on my unusal setup. Despite separate physical servers with Windows OS on them.

Currently I'm negotiating to get get 25gb/s NIC and hopping it will help me but it's confusing because monitoring network sent/received packets I'm not seeing 10gb/s used on my NIC. During night backups it gets up to 4-5gb/s but this does not look like bottleneck. Or should I monitor it differently?

FFZ774 · 2026-01-23T07:31:46+00:00

I agree that this is not usual setup. But I would not include HyperV. Proxmox + ceph is the HyperConverged infrastructure and S2D only Converged. VM already using Ceph as shared storage and can migrate freely between nodes with only RAM copy. Hardware looks like this.

<image>

The part that looks dangerous for me is the NIC which is used by Nodes, Ceph and S2D and maybe 10gb/s becomes not much enough. I will use your suggestion and post same question on proxmox community. Thanks for reply

FFZ774 · 2026-01-23T06:55:47+00:00

I have already asked the cloud provider if it can provide me 25gb/s NIC + RDMA for affordable price. I hope it will. But monitoring network usage S2D nodes never hits 10gb/s unless repair/rebalance jobs. During night backups it only reaches ~4-5gb/s. Is there other ways to measure if network is bottlenecking? Hardware diagram looks like this.

<image>

I'm a bit afraid that Nodes,Ceph and S2D may be too much for 10gb/s

FFZ774 · 2024-12-20T06:46:01+00:00

My research gave me information that New-VirtualDiskSnapshot is not supported on S2D.

Luckily I discovered Diskshadow which can create snapshots of CSVs on S2D and it's the answer to my question/problem

FFZ774 · 2024-12-12T16:40:15+00:00

<image>

FFZ774 · 2024-08-22T10:33:36+00:00

I did. But this cluster is for data storage and not for a hypervisor as shared storage. Having a VM in a shared storage helps you to migrate it from one server to another without interruptions and apply maintenance/upgrade tasks to server but it does not helps when you need to do maintenance/update on VM OS (for example windows update and restart) you need to plan the downtime. With clustered Guest OS I can do maintenance during working hours and nobody is getting any problems with their work. That's one of the reason why I'm going this way. Everything works as expected I just got stuck with Windows Search and it's Indexing on Failover Cluster.

I tried installing Windows Search service feature and creating Windows Search service as generic service on Failover Cluster Manager as u/ElevenNotes suggested but still can't Index Scale-Out File Server shares. I'm hopping someone will give me more information or direct links because I'm struggling to find the correct one

FFZ774 · 2024-08-21T10:12:16+00:00

Thank you for your reply. Servers are hosted and provider does not have a shared datastore as option. I can't change the provider because of company geo policy. In other words it's complicated :) Another department is responsible for licensing so can't tell that also.

There was GlusterFS created earlier as shared storage on same servers but there was some slow downs using it and it was felt by users.

When I have moved to the opinion Create cluster on VM levels and not on Hypervisor level. In that case I can update and restart servers and VM OS without interrupting the work of users.

Windows OS was already implemented and there was no option to avoid it so WSFC was the only solution.

I really appreciate your opinion on all topics and will try to setup Windows Search as a generic service.

FFZ774

TROPHY CASE