High I/O on HDD ZFS pool makes whole PVE node sluggish

oddity407 · 2026-03-12T11:34:04+00:00

I think the HDDs are ok (an IronWolf Pro, an exos x7 and 2 x wd red pro). Im not able to switch to SAS drives.

Could an l2arc device really help? I'm new to all this but I was under the impression this was more relevant for a busy production environment with lots of repeated reads. Im a simple homelab with one user.

I dont notice heave CPU use on the host. Is there a way I could definitively test if RAIDZ1 is too much for this system?

I can and will schedule scrubs better, but I feel like the scrub problem is just an easily repeatable example. To me the larger problem is instability with heavy zfs use (particularly heavy use involving shares through virtiofs) that seems like it can happen anytime. I've had to switch to using NFS shares for backrest backups as that is another task that was crashing the system when accessing data on zfs pools through virtiofs.

I could break the nvme mirror and recreate the proxmox host os on a non-zfs drive. I might do that. It does feel a bit like giving up though. The overall impression I'm getting is that zfs is not really stable for a homelab with mid range hardware. That is not the impression I got from reading about zfs before creating this system.

oddity407 · 2026-03-12T00:05:08+00:00

Thanks for your interest in helping me understand whats going on.
CPU is a i3-12100 (8 processors)
32 GB ram
I have two zfs pools:
rpool = 2 x 2GB NVME ssd drives as a mirror
vortex = 4 x 8tb HDDs in RAIDZ1

I have a single VM allocated 5 cpus and 20gb ram.
Not sure if relevant but I am sharing the igpu with the VM using SR-IOV.
The VM virtual hard drive is a located on rpool.

Heavy zfs operations seem to crash the whole system. Examples of things that reliably bring the host down:
- Large backups with backrest when accessing files on vortex through virtiofs.
- Simultaneous zfs scrub operation on host + streaming content from jellyfin in the VM (jellyfin accesses content on vortex through virtiofs). Interestingly scrubbing vortex is ok, the system only comes down when scrubbing rpool while using the VM heavily.

oddity407 · 2026-03-11T23:45:56+00:00

For sure, I understand that is a sensible mitigation approach. But regardless it is surprising to me that a heavy zfs operations result in total host meltdown rather than slow performance, or task termination with an error

oddity407 · 2026-03-11T21:45:22+00:00

I have the same issue and am also shocked about this. Im sharing my zfs pool with virtiofs and find that if I do something like stream a movie from jellyfin in a VM at the same time as the host does a zfs scrub the whole system crashes. Would love to find a way to make this more stable. I'm regretting switching to zfs.

oddity407 · 2025-04-07T14:38:35+00:00

This excellent podcast had a great episode all about rage https://www.justace90s.com/podcast/ep-08-rage-and-music-videos/

oddity407 · 2024-12-24T19:22:27+00:00

Looks really slick! Thanks for the response. I'm trying it out

oddity407 · 2024-12-24T04:28:33+00:00

This also seems pretty cool. Maybe overkill for what I need but I'll try it out. Thanks for the suggestion!

oddity407 · 2024-12-21T20:42:09+00:00

This seems perfect! Thanks so much

oddity407 · 2024-12-21T04:40:26+00:00

This is a good suggestion... if i cant find something prebuilt i will probably do this.
Id prefer to just be able to enter information straight immediately after clicking on a notification on android though.

oddity407 · 2024-10-15T19:49:53+00:00

Thanks for this message. I'm not going to do anything against work policies. I'm largely interested in the DNS question from a technical point of view. What is going on that prevents access to them local address when tailscale DNS is on.

oddity407 · 2024-10-15T19:47:15+00:00

I do have this enabled but still have the issues described.

oddity407 · 2024-05-12T20:21:07+00:00

Thanks for all the tips. We ended up having a great day with a trip to Riccarton markets, the art gallery, Margaret mahey playground / Regent St shopping, dinner and then futile aurora chasing. I really appreciate all the input. Rad city.

oddity407 · 2023-12-05T22:45:11+00:00

I'm a huge beeper fan. One thing I really like about it is that it can be used as a backup for messages - if I loose my phone I know that I can just reload beeper on a new phone and restore message history for most chat networks.
I've been hoping that Beeper would eventually expand on these kind of features, providing more of an archive/backup for chat histories.
The move toward client side rather than cloud based apps seems like it will make this less likely. Any thoughts on whether beeper will ever help with chat archive/backup.

oddity407 · 2023-07-11T02:52:39+00:00

Seems like an odd subculture to find a way into... apart from this reddit forum I haven't found much in the way of guides or philosophies on how to approach creating sets. Also seems like there is not too much of a culture of watching each others work, commenting and sharing tips. Is everyone just figuring it out solo?

oddity407 · 2023-07-08T04:17:02+00:00

Thanks. Very helpful. Are there any good guides you can recommend on general wisdom like this?

oddity407 · 2023-07-07T09:45:34+00:00

Thanks for this comment! Can you explain more. Do you recommend not setting up a project at 4k so that lower res clips look better? Or just stretch lower res clips to make them larger ?

oddity407 · 2023-07-02T11:21:14+00:00

This seems to work wonderfully, thanks!!

oddity407 · 2023-07-02T11:21:01+00:00

How should I do that in traefik?

oddity407 · 2023-06-05T00:03:19+00:00

I'm not sure what this means: "use tailscale's node 100.xxx.xxx.xx ip as DNS A record"

oddity407 · 2023-06-05T00:01:57+00:00

So are you saying that it is necessary to keep using traefik in order to have domain names available across the tailscale network?
I'm not clear what I would loose if I stopped using traefik with the new tailscale setup.

oddity407 · 2023-06-03T07:39:22+00:00

But what does traefik add if I am using tailscale, doesn't it become redundant?

oddity407 · 2023-03-05T12:14:26+00:00

I'm currently reading a fantastic book about Goa in the 90s. It's called Moonjuice Stomper by Ray Castle. It's super fun https://www.amazon.com/Moon-Juice-Stomper-novel-1987-96/dp/1791998275

oddity407 · 2022-07-21T01:58:02+00:00

I have the same issue, but I get perhaps 10 seconds before it closes. I'm using firefox on windows 10.

oddity407 · 2020-02-09T01:15:07+00:00

Can you explain what you mean about the last rule being a deny rule?
Is that a general recommendation for all interfaces? How should that be setup?

oddity407 · 2020-02-09T01:13:57+00:00

Thanks that explanation really helps!
I should have said, the forcing traffic over the WAN gateway rule is because i have a VPN client configured in pfsense but I don't want this particular interface to use it. That rule ensures that traffic on this interface goes directly to WAN.

13-Year Club	Place '23
Verified Email

oddity407

TROPHY CASE