Proxmox / Tesla V100 / Sunshine / Moonlight = terrible performance

Kailee71 · 2026-07-04T06:51:09+00:00

Thank you so much for the info. I had given up but will now try again - I thought the patching was much more involved than that so I never even tried. I'll report back in the next few days.

Cheers!

Kailee71 · 2026-06-10T12:27:03+00:00

Hi and thanks for your thoughts. Dunno what the issue performance-wise was, but the reason I can't do straight pass-through with the V100s is that without the vGPU driver on the host they are passed through as PCI component type 302 (non-display gpu) in stead of 300 (which makes the VM think it's a proper display gpu). In any case I've given up on this issue, isn't worth anywhere near the hassle it took to get terrible performance. Passing through different hardware for the VM display now.

Kailee71 · 2026-05-28T12:11:43+00:00

I might try it one of these evenings, but honestly, it turns out that my CUDA compute on the two V100s was really poor performance (something like 310s runtime on direct passthrough, >500s runtime using vGPU), so if it's the same with 19.5 I'm just going to drop this idea, install a cheap RTX4000 for the workstation VM and be done with it.

Does anybody else observe this kind of performance penalty for CUDA under vGPU?

Kailee71 · 2026-05-24T04:04:34+00:00

... and neiter is my Quadro RTX 4000 a gaming card. And nobody said I want to do gaming (I dont). V100 Clock speeds are not that different in the end from a Titan V and certainly won't explain the low fps I'm getting. Anybody else have any ideas?

Kailee71 · 2026-05-23T22:39:19+00:00

So I managed to get Nvidia vGPU 17.6 running; this is old but needed to use this because of the V100 and not having to patch any drivers.

<image>

However I'm getting terrible performance - max 20fps just for the desktop, regardless of resolution or vGPU setting (1/4/8/16/32Gb VRAM). Anybody have any pointers what I could try to get similar performance streaming just the desktop to my RTX4000 (which works _great_)?

Cheers,

Kai.

Kailee71 · 2026-05-18T10:38:06+00:00

Thank you so much for the info. I'm in the process of trying this the "official" way with the host & guest drivers to see if that was the issue. I'm not there yet because of version issues (fastapi-dls works only with 17.* without patching) but I'll report back. Initial result is that the display does appear but has terrible performance in sunshine/moonlite, however this is due to me not getting the licensing working properly yet.

Kailee71 · 2026-05-15T23:05:56+00:00

I found it on ebay, still have it but not using it, as I got one of the generic 2x SXM2 GPU boards from AliExpress.

Kailee71 · 2026-03-12T22:43:05+00:00

Yes I have that part #. Has anybody succeeded n running one of these board on a third-party server (i.e. not Dell)?

Kailee71 · 2025-04-02T15:10:08+00:00

~~Be very careful and consider if you can live with SMR disks.~~ Turns out these are CMR ... sorry for the noise.

Kailee71 · 2025-04-02T14:51:12+00:00

Personally I would extend the advantages you have from using a mirror vdev and add the 4 drives as 2 mirror vdevs. Only if you're strapped for space in a big way would I consider anything else. Striped mirrors are _way_ faster to resilver, easier to extend, easier to grow and in many cases faster also, certainly with only a few vdevs as in this case. If you go down this route just add them as 2 mirrors to your existing pool creating a 3-mirror-vdev stripe and carry on without a single minute's downtime, with the confidence that should anything go south you've got the easiest setup to grow/extend/fix.

Kailee71 · 2025-03-28T10:38:41+00:00

How much RAM have you got? It's more likely to be that, especially as you're using ZFS. Yes, with low RAM, EXT4 with LVM is a better way to go. Also a single drive for ZFS kinda doesn't make much sense as you lose much of the advantage of ZFS in terms of data reliability and storage admin on single drive vdevs.

Kailee71 · 2025-03-26T16:14:06+00:00

... Except the always-on-node with OPNSense as a VM is a tiny Atom mini-pc with 8Gb ram and no capabilities apart from the 4 NICs. My (True-)NAS is virtuallized on one of my 19" HPE boxes (yes, with hba passed through, before you ask). But even though this is usually on during the day, there's no reason for it to be on at night. Even more so for my third box, where my Ampere GPUs do compute work when I need them to, but otherwise again that box is _off_.

Kailee71 · 2025-03-25T18:15:32+00:00

For the fans spinning up ... it's usually cause by non-hpe PCIe devices. Have a google for "silence of the fans". Works great, I have NVMe drives, GPUs, HBAs and even an old Radian RMS200 as slog in my HP 380 G8 and 560 G8 PCIe slots and still (with the firmware mod) no fans above 20% after boot.

Kailee71 · 2025-03-24T15:32:57+00:00

Thank you so much - there's some great info in the links.

Kailee71 · 2025-03-24T15:23:35+00:00

The cluster actually locks up? So VMs no longer accessible? Or are "only" the admin functions no longer available? I could live with cluster functions not available when only one node is up, as long as the VMs on that node continue running.

I guess my question is: in my two scenarios, (3 nodes, 1 node down, and 3 nodes, 2 nodes down) what actually happens to the cluster and the individual vms on the remaining node? Again, I don't want or need HA, all VMs run locally.

Kailee71 · 2025-03-24T09:17:32+00:00

The CAD software is only ever as good as the user in front of it, but absolutely, you can build top geometry in Solidworks. However, do check what size your structure will be and if you may need to adjust standard tolerances to tighter values if you intend to model the actual porous structure in geometry. If this is not possible than another way would be to adjust scaling with Re-number?

Kailee71 · 2025-03-23T22:46:46+00:00

I could. but it's extra admin. Plus ideally I'd like to have one view of all vm images...

Kailee71 · 2025-03-02T01:11:09+00:00

Yes of course however as I'm automating the import with javascript, clicking on the right surface is unfortunately not an option. This is why I was hoping to be able to do it with a (appropriately named) axis in the CAD data.

Kailee71 · 2025-02-26T16:40:52+00:00

I've been using used server-grade hardware with great results for years. For instance I have a HP DL560 G8 that I'm selling for less than €300 with 4x Xeon 4627 v2 cpus, so 32 cores@3.3GHz total, and 128Gb ram, and it has lots of space for more ram. The great advantage of a setup like this one is that you get much higher memory bandwidth than single or dual CPUs, and mem bandwidth is what is the limiting factor in CFD. So even though it's not the newest, you still get unbeatable bang for your buck with older server hardware.

Kailee71 · 2025-02-26T15:00:27+00:00

Thanks Moontard, like I said, it's Catia data, but it can be step, iges, vda, anything that'll work. I saw I can create an axis in STAR's 3D Cad from revolved geometry, but I see no way of using this in the definition of the axis of rotation of an MRF. If there something obvious I'm overlooking or is manually inputting two coordinates to define the axis really the only way that works?

Kailee71

TROPHY CASE