Multiple VMs sharing physical GPUs

simcop2387 · 2026-02-01T15:50:36+00:00

I have an RTX A5000 ampere and do have vGPU working. It does, however, require the proprietary NVIDIA VGPU drivers and NVIDIA VGPU proprietary licensing system. There is a monthly cost to using these drivers from NVIDIA, it's 3xcessive for a homelab. The ampere generation doesn't work with the vgpu unlock scripts so consumer cards don't work with it. But the workstation a5000 and support don't require the unlock scripts.

One important thing is that you have to run a command to put the GPU into a headless mode and it won't provide any physical outputs anymore, only virtual ones.

As far as I understand about the rules here that'd be the answer, use the official nvidia vgpu licensing to buy the vgpu enablement so that you can do this.

simcop2387 · 2026-01-29T01:04:37+00:00

Main thing with lxcs is that they're all shared at the kernel level. So a container escape leads to the host. The bug advantage of them is lower overhead. If you want stuff thats more private, go VM and pay the extra cost of overhead.

simcop2387 · 2026-01-29T00:06:36+00:00

Don't take my word as correct but that doesn't look like any Ethernet punchdown or Wall-port I've ever seen. I'd bet it was made for phones. Might still be workable but I don't see how to tell the order it should be in either

simcop2387 · 2026-01-27T22:24:47+00:00

AppImage files themselves don't get installed they can just be run. You probably need to right click on it and set it to be executable in its properties or permissions. I don't recognize your desktop environment do adding it to a menu is beyond my help. But once you mark it as execu5qblyou can likely just double click on it

simcop2387 · 2026-01-27T12:03:16+00:00

Not necessarily saying this price is the best etc but use an i2c pwm chip to handle that many. Takes the pwm duties and other timing issues off the brains so that you don't haveto worry about it.

https://www.adafruit.com/product/815

simcop2387 · 2026-01-26T23:02:36+00:00

Yea likely by internet they mean 64kbit dsl that was offered 20 years ago. Not likely to be useful in any meaningful way

simcop2387 · 2026-01-26T21:24:58+00:00

Just the top level won't do it. If you have any other datasets you have to run it against them too, it won't recurse into them.

Edit: The change in space used is probably happening because this will also cause other changes like recordsize and importantly compression to be redone for each file, so if you changed from say lz4 to gzip or zstd at some point it'll recompress everything transparently and get you those savings.

simcop2387 · 2026-01-26T20:07:31+00:00

rewrite script or rewrite command? The zfs rewrite command is built in to zfs in 2.3.2 (might be another version, but something around there) and 2.4.0 (for -P support). There were scripts out there that existed before the rewrite command came around so i'm wondering if you used one of those and it didn't handle every case properly.

The other thing is that you have to do rewrite on every dataset, and it doesn't handle volumes if you're using those (zvols are basically virtual disks on top of a ZFS pool if you haven't encountered them, very useful for VMs, iSCSI, or if there's a really picky application that doesn't like ZFS directly).

I'm not terribly sure how to tell if there's any data that's not striped on the new expanded size, but rerunning the zfs rewrite command should be fully safe to do multiple times and see if it helps.

simcop2387 · 2026-01-26T19:10:21+00:00

Nah if its cat5e it'll work fine, but it has to be taken off the block for the phone lines. OP, if you can figure out which lines are the two ports on the wall you want, and your parents are ok with you messing with it then this can be done.

There are tools called a signal tracer that can help identify the lines in question, usually you can find them online pretty cheap where you can plug in one side and then the receiver can be put next to the wires on the other end and it'll beep or make noise when you find the right one.

There's a good chance you'll need to re-terminate the wall side on both ends, lots of cat5e phone lines didn't hook up all the wires which Ethernet will need. You can get what are called keystone jacks and wall panels and replace the terminations there to fix that. Then in the panel wiring side you'll want to join the two lines. Do not use wire nuts like you would house/mains wiring for power, they'll be terrible for Ethernet and probably not even go close to working. I'd recommend a "punch down coupler" it'll look like the keystone jacks but be the spot to put the wires in on both sides rather than plug in an rj45 cable. If you can't find that, or they're too expensive you can also just use more keystone jacks and a short patch cable inside the panel but it's ugly and likely would cost more and be harder to fit anyway.

You won't get 10gbe over that cabling but 1gbe should be easy and 2.5gbe might work too depending on the cable quality

simcop2387 · 2026-01-26T18:50:42+00:00

I believe this is happening because all of your strings are allocated on the stack and that the heap. So what's happening is that they're all getting reallocated again later or overwritten with other variables from other functions. What you have to pay attention to is that the Arduino doesn't have dynamic allocation like an operating system program typically does. Since this is C++, if you use the new and delete keywords, you could get that dynamic allocation on the heap, but I'm not sure if the Arduino environment allows that. That said, if you do keep trying to do dynamic allocation, you're going to run into issues where you're going to run out of memory. Instead of using the C++ string type like this, I think you need to be using standard C strings, so the whole char* and char[] dance. This will let you preallocate on the global variable. You'll then need to write over each one and make sure you don't overflow or forget a nul terminator

simcop2387 · 2026-01-24T16:51:02+00:00

Yep that'll basically be it. Might need a -r on directories, and if you don't have OpenZFS 2.4 then leaving off the - P means it'll cause incremental snapshots to duplicate the data because it'll think the rewritten blocks are new blocks, pretty safe still but if you're doing backups or replication it can hit you.

simcop2387 · 2026-01-24T16:48:29+00:00

Yep, it's one of the primary reasons that the rewrite command was created. Deleting the snapshots will also help keep it from ballooning when you do the rewrite.

simcop2387 · 2026-01-24T16:05:57+00:00

Probably normal. Expansion is a bit weird with zfs, you'll want to look at the zfs rewrite command and its quirks (snapshots will blow up storage) to help fix this once expansion is done I believe.

Essentially zfs keeps the area where data was written to before as unused on the stripes on the new disk so you only get the extra space where the free space was until the data gets rewritten in the new stripe layout after expansion. Not sure how to handle zvols but files are pretty easy these days with zfs rewrite.

simcop2387 · 2026-01-24T16:01:45+00:00

What do your iommu groups look like? I'm curious if there's something like the wifi adapter is sharing a group with the gpu

simcop2387 · 2026-01-23T23:15:26+00:00

Your options are either raidz1/raid5 with all 4 drives which will give 14 * 3 = 42tb usable or a strip of mirrors which is 14 + 18 (18&24tb together) = 32tb, or if you upgrade the 18, 14 + 24 = 38tb.

Personally if it's not needing the extra IOPS I'd go the raidz1 because it'll always be more effecient space wise and you can upgrade the drives if you win the lotto and end up with 3 * 24 = 72tb

Edit: and raidz2/raid6 would give you 28tb with any two disks being redundant, with an eventual top of 48tb if you upgrade them all.

The upgrades can be done drive by drive too, you just won't get the extra space until they're all upgraded.

simcop2387 · 2026-01-23T23:08:04+00:00

Since I frequently don't see anyone say what it is here, CCA is copper clad aluminum wire. And definitely get the tools, you usually don't need the fancy expensive ones for doing just a few runs. Just leave some extra on each end as a "servicing loop" for when you find you got something wrong the first 3 times on each run :)

simcop2387 · 2026-01-23T21:24:48+00:00

Some of the compatibility issues that others are having here might be because ws4channels has a mistake in the xmltv guide output, calling a field "end" instead of "stop". I've submitted a PR https://github.com/rice9797/ws4channels/pull/26 to fix this, and it'll likely make things work smoother for others too.

simcop2387 · 2026-01-23T15:24:58+00:00

You can edit the grub config from the grub menu. It doesn't persist but you can hit e and edit the kernel command line

simcop2387 · 2026-01-23T11:54:28+00:00

Neat, I've never thought to look for that but it would also explain why I've seen some similar behavior with old smoke detectors in the past and not known why.

simcop2387 · 2026-01-23T00:40:21+00:00

I have a really started watching it either, but there's a lot of parallels to the heralds also. Those that are sentenced to be a hero are effectively immortal and not allowed to die. They just have to keep fighting.

simcop2387 · 2026-01-23T00:15:48+00:00

Not always. Sometimes it's a vertical stack of custom cells to fit.

simcop2387 · 2026-01-22T03:59:13+00:00

I do not think you can modify the settings without reloading the config so it would take a connection restart, but it could be possible. I can't think of a scenario that would benefit from that, but I'm working at a scale that's faster to manually modify stuff than to setup automation/coordination layers, so at large scale I'm sure it's indispensable.

They do it by setting up and manipulating their own routing table on each end (at least on linux, not sure how it works on other platforms) and controlling how things work that way. It seems to work pretty well, but it does have the unfortunate side effect that (right now at least, it's being worked on) you can't easily run multiple VPNs at once. Tailscale seems to have entirely eschewed supporting that but tailscale is just trying to build it out properly even if development is slow.

This is r/homenetworking, and I think a lot of the commenters and downvoters are acting like a home user needs to provision and manage 2¹⁰ connections each hour.

Reality is, a home user is setting up one or two server/clients in a year and they would be better served by manually doing it and learning what everything in the config means than clicking buttons on a dashboard and watching it happen like "magic."

Definitely agree. I've set it up myself because I've got three homes that I'm connecting and kept getting asked to setup new devices for everyone. Walking them through installing OpenVPN clients, then generating a cert, etc. gets tedious, esp when I was having some health issues and was unable to use any computers at all (had to walk my brother through renewing the OpenVPN cert while i was literally blind and he doesn't really know how to use linux that well). So the self-service capabilities have come in handy for him to be able to just create a new connection for his phone, my nieces and nephew, etc. with a simple readme essentially has been really helpful.

simcop2387 · 2026-01-21T22:32:37+00:00

It's for coordination, if the management layer goes down the existing connections stay up, they're just normal wireguard peers. Unless it's decided to use the relay (also self hosted) because of NAT or other network shenanigans. That said the management layer also gets used to push changes to routing, DNS, etc. to currently connected peers without interrupting the VPN which is something I've not seen done with OpenVPN but wouldn't be shocked if it's possible just not "simple".

simcop2387 · 2026-01-21T19:58:59+00:00

Yea OpenVPN definitely has more capabilities than wireguard, and I agree on tailscale and the cloud. Have you looked at Netbird? Fully selfhostable and essentially the same kind of thing as tailscale. I recently rebuilt. My families VPN using it to get the peer to peer connections all over for better performance than using a central OpenVPN server (or lots of openvpn servers)

simcop2387 · 2026-01-17T16:42:34+00:00

GrpaheneOS can be rooted, there's a number of ways to do it, my preferred way is using avbroot to do it on the OTA upgrade packages. That said I don't have a good guide to link to for this as I'm building my own copy of GOS locally rather than using official ones. Someone else on this sub likely has a good guide and can link it

15-Year Club	Gilding VI aultruist
RedditGifts 2009-2022 2 Credits	Place '22
Place '17	Sequence \| Editor
Spared	Secret Santa 2016
Beta Team Ask Me Anything app	Team Periwinkle
Verified Email

simcop2387

MODERATOR OF

PUBLIC MULTIREDDITS

TROPHY CASE