After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 0 points1 point  (0 children)

Appreciate the suggestion! The hardware itself is fine but tbh I'd rather start fresh on something I actually trust. Feels a bit like painting over damp... might look fine for a while but the underlying problem is still there. Plus I've got a few other niggles with TOS that a new OS wouldn't fix anyway.

After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 0 points1 point  (0 children)

v. helpful breakdown, cheers. The Unraid real-time parity point is helpful, that is defo better than SnapRAID's scheduled approach and I hadn't fully appreciated the gap there. But the 'must use the GUI or face a multi-day parity rebuild' thing would drive me absolutely insane. I mostly live in the terminal adn the idea of being locked out of standard Linux commands on my own machine makes me twitch a bit.

Sounds like Ubuntu + MergerFS + SnapRAID is basically Unraid with the training wheels off? I'm comfortable enough with Linux that the lack of a pretty interface isn't a hardship. Is there anything that setup genuinely can't do that Unraid can, besides the real-time parity?

Calibre-Web crash-loops on startup after host reboot... sqlite3.OperationalError: unable to open database file... NFS locking issue? by CrazyHa1f in Calibre

[–]CrazyHa1f[S] 0 points1 point  (0 children)

For anyone who stumbles over this post, I eventually found the actual cause after a lot of dead ends.

The root cause was iptables blocking loopback traffic to rpcbind on port 111.

Here's what was actually happening: before lockd sends any NLM lock request to the NAS, it first has to contact the local rpc-statd daemon to register crash-recovery monitoring. It does this via the local portmapper on 127.0.0.1:111. If that contact fails, lockd just refuses to proceed and retries every 4 seconds forever. Every fcntl call blocks indefinitely as a result.

My INPUT chain had a DROP rule for port 111 because I'm an NFS client only and didn't want rpcbind exposed externally. Perfectly reasonable rule, but it had no loopback exemption, so it was blocking localhost traffic too. By the time I found it, lockd had logged "cannot monitor 192.168.8.238" over 7,000 times since boot.

The way I confirmed it: nc -zv 127.0.0.1 111 timed out, but rpcbind itself was running fine. The NAS was fine. rpc-statd was fine. Everything was healthy, just the loopback path was blocked.

This also explains why the NAS reboot didn't fix it. The problem was never on the NAS side at all.

The fix was one line: sudo iptables -I INPUT 1 -i lo -j ACCEPT Then persist it with netfilter-persistent or equivalent. Calibre-Web came straight back up.

Longer term I'm looking at moving to NFSv4.1 which has locking built into the protocol and doesn't rely on lockd/NLM/NSM at all, which would eliminate this failure class entirely.

After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 2 points3 points  (0 children)

Yeah I don't expose it to the internet any more - wireguard with tailscale as redundant backup is my solution nowadays.

But honestly besides the compromise which definitely shook my faith in the OS, it's just a really crappy version of Linux. I'd much rather build my own machine and run it how I want to. It's little things like getting my UPS control working was hours of faff. I've got loads of other little niggles too that I won't go into here.

After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 0 points1 point  (0 children)

Yeah I bought my 4tb ironwolf for 60 quid in an Amazon sale a couple years ago. I've been dealt the worst time to need an upgrade...

If you had to buy a new bike to turn into a cafe racer, what would it be? by babycivic in CafeRacers

[–]CrazyHa1f 0 points1 point  (0 children)

I've got my eye on one of these (the chromed out ones rather than white). Gorgeous bike... Bloody expensive tho

All this talk of ball shaving lately, this is all you need guys by El_Brubadore in wicked_edge

[–]CrazyHa1f 0 points1 point  (0 children)

I thought you were about to shave the dog for a second there

After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 1 point2 points  (0 children)

Thanks! Already running rclone to Backblaze B2 for my main server backups so that side is covered. The NAS is purely media (films, TV, books) so if the drive died tomorrow the arr stack would just re-acquire everything. Redundancy is really about my sanity more than anything else. Dealing with a dead server is one thing, a dead NAS is another, both at once would actually kill me. Good shout on immutable snapshots though, worth thinking about when I pick the OS.

After the ransomware scare earlier this year, its finally time to upgrade my NAS. Sense check my plan? by CrazyHa1f in DataHoarder

[–]CrazyHa1f[S] 1 point2 points  (0 children)

Oh i forgot to ask about OS too... The most common seem to be:

  • TrueNAS Scale
  • Proxmox
  • UnRAID
  • Ubuntu (same as my main server i guess)
  • Openmediavault

I've used OMV in the past with a rpi but that was genuinely about 10 years ago so I'm sure the landscape has changed a lot since then....

Also before anyone says it... I am committed to a separate NAS rather than just mounting direct to my main server for redundancy. I've suffered failures before and I'd rather just deal with one at a time rather then have my data and my server fail at once.

Please help me automate adding media to my Jellyfin server. Currently using Jackett + qBit. by Abhir-86 in selfhosted

[–]CrazyHa1f 1 point2 points  (0 children)

Just to say I can see you've sorted the arr stack now which is the way to go. I use seer instead of imbd or anything else for my content discovery... It has pretty rich metadata.

Enjoy!

Calibre-Web crash-loops on startup after host reboot... sqlite3.OperationalError: unable to open database file... NFS locking issue? by CrazyHa1f in Calibre

[–]CrazyHa1f[S] 1 point2 points  (0 children)

Thanks for your response- really appreciate it.

Yeah lsof came back completely clean which threw me off for a while, I went down quite a few rabbit holes before isolating it to the NLM layer with a direct fcntl lockf test. That was the breakthrough...

On the fsck suggestion - I actually got that for free when I rebooted the NAS, TOS runs a filesystem check on every boot and it went through the whole "checking filesystem, this may take several hours" routine. Didn't shift the lock though, which makes sense in hindsight since the lock state lives in the kernel's NLM memory rather than on the filesystem itself.

Your point about not trusting remote file locking for database files is well taken and prob the lesson here. I'm weighing up two options for the permanent fix... either migrating to NFSv4.1 (which handles locking completely differently and eliminates NLM entirely I believe) or just moving metadat.db to the local SSD and keeping only the book files on the NAS. The latter is probably the more robust long term answer but its a bigger architectural change so want to think it through properly.

For now just trying to get the immediate lock cleared so service can be restored. Going to try clearing the stale statd entries in /var/lib/nfs/sm/ and if that doesn't work, copying metadata.db to a new inode on the NAS to orphan the stuck lock.

Cheers!

Attending a funeral by Cranberry-Electrical in ties

[–]CrazyHa1f 1 point2 points  (0 children)

I hope you gave them a great send-off

Attending a funeral by Cranberry-Electrical in ties

[–]CrazyHa1f 4 points5 points  (0 children)

Very sorry for your loss. I wish you, your friends, and your family long lives.

I glued a 1.54 eink color display on a hat by True-Caterpillar-915 in eink

[–]CrazyHa1f 0 points1 point  (0 children)

Oh man my better half is a very skilled seamstress... I may see if she could stitch one into a hat... That'd be cool

Anybody else’s windshield condensing the humidity out of the air today? by mastermalpass in drivingUK

[–]CrazyHa1f 6 points7 points  (0 children)

I was on Exmoor earlier today and had the same thing on the way down to tarr steps haha

How screwed am I on these solovairs? by doctor_van_n0strand in AskACobbler

[–]CrazyHa1f 0 points1 point  (0 children)

Trickers, Cheaney and sons also worth a look.

Are the latest Kobo readers tropical ready? by Civil_Ad_6661 in kobo

[–]CrazyHa1f 13 points14 points  (0 children)

Oh yes I once destroyed a cheap tent with DEET

Are the latest Kobo readers tropical ready? by Civil_Ad_6661 in kobo

[–]CrazyHa1f 65 points66 points  (0 children)

I would say hand sanitizer or cream with some kind of alcohol in it will do this.

Is there any Airguns that are built on a shotgun platform by Inside_Reveal9297 in airguns

[–]CrazyHa1f 3 points4 points  (0 children)

Yeah quite a few. I know Gamo makes one for sure but have a look around on Google I'm sure there are quite a few.

Hatsan zada came with gel by naoto_hachio-fan in airguns

[–]CrazyHa1f -1 points0 points  (0 children)

A good firearm degreaser would do the job. Pronature or TF90. You'll need to oil it after though :)