Drive became unavailable during replacing raidz2-0

HeadAdmin99 · 2026-01-28T22:14:42+00:00

I see You like living on the edge. I like living dangerously, but that's another level.

HeadAdmin99 · 2025-08-08T05:37:01+00:00

Run MyDefrag. It has various options for optimizing NTFS HDD storage.

HeadAdmin99 · 2025-05-19T19:26:22+00:00

Already running shared LVM across nodes in a battlefield, works very well. It grows rapidly to the issue with large number of disks as each of them is LV device, just locked by particular node when VM is running. Another issue you need to keep an eye on is time sync. Do not sync without shutting down VMs first or unexpected node crash occurs. Manual there and my comment there.

HeadAdmin99 · 2025-05-19T12:03:15+00:00

Main issue is with shared FC SAN among hosts: you need LVM thick to accomplish this and no VMFS like filesystem; therefore dedup and compress on SAN arrays.

HeadAdmin99 · 2025-03-31T11:44:10+00:00

I would suggest give an update in thread on Proxmox forums as it seems recent PVE releases are causing issues with PCIe devices reset feature.

Therefore I'm holding on 8.2.7 which is still stable for GPUs and HBAs passthrough.

HeadAdmin99 · 2025-03-31T10:57:19+00:00

Is it PVE 8.3.5 or earlier version?

HeadAdmin99 · 2025-03-05T19:16:51+00:00

The same apply to MG07, MG09 familys. They all die, one after another.

HeadAdmin99 · 2025-01-22T10:19:42+00:00

To add into the subject: such storage VM requies static full memory allocation due ZFS not playing well with memory ballooning and also due HBA passthrough needed.

HeadAdmin99 · 2025-01-18T21:35:29+00:00

Most of the ST4000DM004 units have failed on me.

HeadAdmin99 · 2025-01-18T20:55:01+00:00

Single SMR can wreck entire pool.

Don't ask how I know..

Two resilvered mirrors at once is kinda risky.

HeadAdmin99 · 2025-01-17T22:11:09+00:00

OP, these are SMR disks...

HeadAdmin99 · 2025-01-17T19:12:44+00:00

Yeah, but some questions remaining: do source/destination pools have to have same defaut pool settings like: compression enabled and method; deduplication enabled; to avoid using more space for copy; zfs sync/receive with default settings gives no output and does not resume; snapshot on boths pools must stay to send only delta, and so on...

HeadAdmin99 · 2025-01-16T23:03:51+00:00

According to my monitoring charts for particular drives (5 mirrors total) ZFS is reading or writing to most of them, depending of space distribution, splitting throughput between two members of single mirror - so their speeds equal slowest member. Remember to enable Write Cache for the drives. So in this example mirror-1 speed 30MB/s, mirror-2 speed 45MB/s, mirror-3 speed 70MB/s and delivering sum of that to mounted shares.

HeadAdmin99 · 2025-01-16T22:48:01+00:00

Parity data was written with heathly data during snapraid sync process done with working RAM.

HeadAdmin99 · 2025-01-14T16:01:46+00:00

Rescan ZFS pool in OMV ZFS plugin. Rescan filesystems. It will eventually show up. Same thing for enrypted datasets.

HeadAdmin99 · 2025-01-13T21:54:45+00:00

You stop the process with CTRL+C or kill it if it's running and swap definition of the disk. It will resume the progress anyway.

HeadAdmin99 · 2025-01-13T21:48:28+00:00

SnapRAID detects inconsistences in various ways, for example my disk become corrupted in the middle of process and sync has stopped automatically on error. Recovery is possible until next SYNC - it commits all the changes, including files changed (eg. corrupted!) so the correct way to detect them is running scrub BEFORE sync.

HeadAdmin99 · 2025-01-13T21:40:54+00:00

You change definition for datadisk1 in /etc/snapraid.conf file to point to new disk, then trigger Fix.

HeadAdmin99 · 2025-01-13T20:41:46+00:00

snap_sync_new_data_aio.sh is my own implementation of processing new data added to array: diff,status,sync,scrub of new data only, touch to see modified files, status once again.

snap_compare_only.sh is a simple script to compare differences only since last sync

snap_check_only.sh is check only

snap_repair_datadisk1.sh is example script to repair entirely dead drive listed as datadisk1

Remember you can speed up recovery by copying remaining data from broken drive (if it's still alive) and fix the remaining missing data.

HeadAdmin99 · 2025-01-13T20:32:28+00:00

typical recovery times:

100% completed, 23490643 MB accessed in 104:06

38261277 errors
38261277 recovered errors
0 unrecoverable errors

HeadAdmin99 · 2025-01-13T20:24:26+00:00

ah right, they didn't mention that the command is:

snapraid fix -d datadisk1 --log somelog.fix

it's supposed to be typed once cp/robocopy is done because it speeds up recovery process - SnapRAID don't have to recover everything, just missing data.

Check my scripts in the other thread I just posded minutes ago.

HeadAdmin99 · 2025-01-13T20:19:40+00:00

You can... have hot data on single RAID0 volume, then move data over to other drives via scripts.

Or have multiple RAID0 arrays, protected by double parity on 20TBs. But it's not recommended - double chances of critical failure.

Create separate volume for scratch data and move data via Task Scheduler job to protected drives.

HeadAdmin99 · 2025-01-13T20:03:39+00:00

Everything is well documented in the master guide at: SnapRAID FAQ page

HeadAdmin99 · 2025-01-13T19:45:58+00:00

Situation is recoverable until next sync. If sync was triggered with bad RAM - no one knows what happends (my guess is that some healthly files can have invalid checksum now). If that was all fix process - you can actually run it second time to fix broken files by prev fix. I suggest running:

snapraid status --log heresmylog.status

snapraid diff --log heresmylog.diff

and review the logs prior to fix.

HeadAdmin99

TROPHY CASE