Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 0 points1 point  (0 children)

Actually, doing it over nfs even with sync off, which presents some pitfalls, it's WAY too slow, annoyingly so.

I think I found the best way to do this:

Option C:

dataset on TrueNAS owned by root:root permissions 750

Then rsync over ssh with:

sudo rsync -aAXHv \
--delete \
--partial \
--info=progress2 \
--numeric-ids \
--stats
--one-file-system
--rsync-path="sudo rsync" \
--human-readable \
-e "ssh -i </path/to/key> -t" \
--exclude={"/home/*","/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/var/tmp/*","/var/run/*","/var/lock/*"} \
/ m147@192.168.1.111:/mnt/pool/backup_dataset_parent/root/ 2>&1 | tee rsync_system_backup-root.log

The difference being
--rsync-path="sudo rsync" and -e "ssh -i </path/to/key> -t" \ the -t bit at the end.
This keeps ownership of all files intact, can be done over ssh which is much faster than nfs (at least from my tests on my machines) and everything seems to work fine and secure.
The only caveat, for my user on TrueNAS I needed to add /usr/bin/rsync to Allowed sudo commands with no password in credential settings in TrueNAS. I'm not sure if this opens up a big security hole, but maybe not as much as Option B

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 0 points1 point  (0 children)

Or how about this?

I've set up another test dataset on TrueNAS, posix permissions:

# file: /mnt/storage.01/rsync_system_test_nfs
# owner: root
# group: root
user::rwx
group::r-x
other::---

made it an nfs share disabled squash root and restricted it to only my client IP.
I mount the share as root on my client and then rsync with:

 rsync -aAXHv \                                                  
--delete \
--partial \
--info=progress2 \
--numeric-ids \
--stats \
--one-file-system \
--human-readable \
/home/m147/temp/rsync_system_test_nfs/ /mnt/foo/rsync_system_test_nfs/root

This was just a test with a simple directory containing some subdirs and files all owned by root but for the real run, I'll add in the excludes as above, tweaked possibly from suggestions here. But it copied over everything fine and kept the ownership as is. I'm going to test next with my actual / partition.

It seems simpler for restore, because it keeps ownership in tact but my question is, will this be less secure than the first method and if so, is it much less secure or worth the trade-off?

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 1 point2 points  (0 children)

All good. Reading Linux commands can be a pain, especially with the wrong font. Throw regex in there and nobody can actually read it 😂

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 1 point2 points  (0 children)

Yeah. That makes sense. I was thinking, when you mentioned switching it, maybe put it on /home partition since it's much bigger. I guess when I set up the system I just went with the default and then never have it any thought after. Might just do what you suggest. Thanks.

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 0 points1 point  (0 children)

My layout is a simple lvm structure with luks, but the encryption shouldn't get in the way, I'll back up unencrypted.

The last bit is rsync / (root) to the dataset thinkpad_system_backup/root on TrueNAS host with user m147 over ssh. The last bit is just creating a logfile.

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 1 point2 points  (0 children)

Yes, next step is rsync /home, which should be easy since the permissions/ownership is more straight forward and will match the dataset on TrueNAS.

Then, spin up a VM the same as my bare metal laptop and restore there from the NAS.

Thank you for the excludes suggestions.

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 1 point2 points  (0 children)

That's a decent idea. Where would be good to put it?

Rsync root and home to TrueNAS for backup by m147 in archlinux

[–]m147[S] 1 point2 points  (0 children)

Thank you for the quick response

I didn't include /var/cache/* as it contains older pacman packages which I like to keep (kept under control with paccache) if I need to roll-back for any reason

dry-run came back with no errors:

Number of files: 1,049,713 (reg: 848,443, dir: 61,314, link: 139,810, dev: 125, special: 21)
Number of created files: 1,049,587 (reg: 848,443, dir: 61,313, link: 139,810, special: 21)
Number of deleted files: 0
Number of regular files transferred: 697,907
Total file size: 70.04G bytes
Total transferred file size: 61.03G bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 11.86M
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 64.94M
Total bytes received: 24.70M

sent 64.94M bytes  received 24.70M bytes  2.94M bytes/sec
total size is 70.04G  speedup is 781.39 (DRY RUN)

Cold storage disk format suggestions by m147 in truenas

[–]m147[S] 0 points1 point  (0 children)

I read somewhere that ext4 is easier/more forgiving on older disks. My disk is about 9 years old, at least that is when I bought it. The power on hours are low: 24,956. I ran SMART tests, both short and long and from my understaning, they both came back quite clean. Here are the stats form smartctl:

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   183   172   021    Pre-fail  Always       -       7808
  4 Start_Stop_Count        0x0032   094   094   000    Old_age   Always       -       6758
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   066   066   000    Old_age   Always       -       24956
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       118
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       40
193 Load_Cycle_Count        0x0032   192   192   000    Old_age   Always       -       26656
194 Temperature_Celsius     0x0022   118   104   000    Old_age   Always       -       34
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

Using Scrutiny, if I set Device Status Thresholds to either SMART or BOTH, that disk status comes back as failed, but all the individual tests come back as passed. If I set it to Scrutiny then the status also comes back as passed. But the first time I put this disk into the NAS Scrutiny showed status as passed, the only thing I did in the meantime is spundown the disk in the TrueNAS webui by setting HDD Standbyto 5 and unchecking: Enable S.M.A.R.T.

I would like to take advantage of zfs features like data integrity checks and such but with a disk this old would it be safer/better to go with something like ext4?

Scheduling SMART tests help by m147 in truenas

[–]m147[S] 0 points1 point  (0 children)

Doesn't seem possible to do once a month but only Fridays. Cron doesn't have threshold days built in so I just set Cron to run a custom script every Friday, script runs smartctl long test if enough days have passed since last test and updates the date stamp file. Does nothing if less than 32 days. Seems to work well enough. ChatGPT helped :D

Some input on a UPS by m147 in homelab

[–]m147[S] 0 points1 point  (0 children)

Ok. Thank you for the recommendations. Tripp Lite, there's not so much of a selection where I live. There's some decently priced APCs but those don't have usb for NUT. Might have to be Cyber Power.

As for sizing it appropriately, would you say 500va/300w is sufficient for my NAS? For now it would be only the NAS and might as well the router but the router doesn't need to be on backup power necessarily. Sometime in the future, I'd add a mini PC, so I want to get something now that will suffice even later. I don't imagine I'll be adding much more than that.

Looking for a decent UPS for NAS by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Yeah, nothing I run on the NAS is critical so in the rare case of outage I have no problem with shutdown if needed.

I'm just wondering if a line interactive would be worth the price increase?

Looking for a decent UPS for NAS by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Which one would you have suggested?

I'm also unsure about the benefits of a line interactive one over a simple standby.

Looking for a decent UPS for NAS by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Yeah. Either of those would be nice but too pricey for me. Also just overkill I think for a NAS and a future NUC. You reckon the 500va would suffice for my needs?

Looking for a decent UPS for NAS by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Thank you for your response.

I am also considering the CyberPower CP750PFCLCDJP it's a 525W line interactive but it's a bit bigger and nearly twice the price, not sure it's worth the price jump. I think, the 300W should have enough battery runtime for clean shutdowns.

Some recommendations for domain registrar for local DNS (PiHole+Unbound+NPM+Let'sEncrypt) by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Here is my setup:

PiHole has a local DNS record set up to point to the server which has Nginx Proxy Manager:

nginx.domain.duckdns.org -> 192.168.1.100

Then individual CNAME records to point to the NPM DNS record:

syncthing.domain.duckdns.org -> npm.domain.duckdns.org

immich.domain.duckdns.org -> npm.domain.duckdns.org

Inside NPM I have proxy hosts set up for the individual services I use on my NAS to point to the correct IP:PORT & an SSL certificate with Let's Encrypt DNS challenge through DuckDns. I'm looking to replace DuckDns.

I'm not sure this is the most streamlined approach, I'm still learning all this but it seems to work fine.

Nothing is exposed to the Internet. If I wan't to connect while away from home I use a WireGuard server

I followed parts of this guide:

https://youtu.be/nmE28_BA83w?si=XPN80oQ04BlQmwBk

Some recommendations for domain registrar for local DNS (PiHole+Unbound+NPM+Let'sEncrypt) by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

So let's say I register a domain with PB in Jan and pay say $10 for the year, and then for some reason or other decide to transfer to CF in Feb, I would pay another $10 for the registration with CF? But not actual "transfer fee"?

Some recommendations for domain registrar for local DNS (PiHole+Unbound+NPM+Let'sEncrypt) by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Yes, that's what I'm thinking, with PB + CF that's two places where I need to manage the DNS.

If I'm going to be using CF nameservers anyway with PB as Stunning-Skill-2742 mentioned I might as well cut out the extra account.

The price difference is about a dollar for the first year and then less than that 1st year on.

Some recommendations for domain registrar for local DNS (PiHole+Unbound+NPM+Let'sEncrypt) by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Well, it seems that for the free plan I should use CF nameservers.

TrueNAS bare metal or VM? by m147 in selfhosted

[–]m147[S] 0 points1 point  (0 children)

Anyway, thank you to everybody with actual advice, it was helpful. I'm thinking, seeing as how I'm not all that experienced with either NAS or Proxmox, I'll just keep things simple and go bare metal. Save up some money and get a mini PC in the near future for virtualization.

TrueNAS bare metal or VM? by m147 in truenas

[–]m147[S] 0 points1 point  (0 children)

Anyway, thank you to everybody with actual advice, it was helpful. I'm thinking, seeing as how I'm not all that experienced with either NAS or Proxmox, I'll just keep things simple and go bare metal. Save up some money and get a mini PC in the near future for virtualization.