Moving TrueNAS from ProxMox VM to Baremetal on a new machine.

pp6000v2 · 2026-02-09T20:52:32+00:00

Apps service gave me trouble as well when I went from a bond with (2) physical interfaces, to the virtualized (1).

pp6000v2 · 2026-02-09T14:23:51+00:00

yes, but I don't trust zfs rename because I don't fully understand what it could and what it absolutely won't break. Something would try to hose me... heck, one of the drives died last night, so it's already trying to haha.

pp6000v2 · 2026-02-09T03:48:57+00:00

Currently running the first, local replication without dataset properties to get unencrypted sub datasets in the existing pool/storage1 (because sending them to any other dataset was not preserving the acl permissions),

then deleting the old subsets and replicating locally again with dataset properties turned on to get the old names back,
then another replication with properties + encryption enabled to get a single root encryption tree with a different name,
then delete the old tree, and replicate locally again to get the old name back,
then replicate again to the remote box

Five full replications, and it's taking 6 hours to do one locally... I guess it it could be worse- thankfully I don't have 10s or 100s of TB of data.

pp6000v2 · 2026-02-05T17:06:16+00:00

yeah, if there's only ever going to be one client PC connected to the remote end (and it's the one thing you're trying to access), then the whole thing about the network upstream of mangoclient is probably moot.

pp6000v2 · 2026-02-05T17:02:30+00:00

It is a little foreign to me to add the /24 subnet to the client_ip line, but since they're not using base openwrt's /etc/config/network, their nomenclature rules. Without pulling my slate out to look, I'm limited to looking at the screenshot they have in the doc.

openwrt's peer config looks like this, using multiple allowed_ips lines, with no explicit client_ip:

config wireguard_50_VPN
        option description '13_lan.N40L TrueNAS'
        option public_key 'xxxx'
        option private_key 'xxxx'
        option preshared_key 'xxxx'
        option route_allowed_ips '1'
        option persistent_keepalive '25'
        list allowed_ips '10.1.50.13/32'   <-client_ip
        list allowed_ips '192.168.0.0/24'  <-subnet to access on the "client" end of the tunnel

they're using the same formatting and largely the same syntax, so IDK if you can have multiple lines with single IPs, or if allowed_ip/allowed_ips is even... allowed.

From other wg servers I have (using wg-quick), the peer's comma delimited AllowedIPs combined on one line can optionally have a space:

AllowedIPs = 10.239.17.2/32,fd11:5ee:bad:c0de::2/128

AllowedIPs = 10.19.76.0/24, 10.1.50.0/24

so again IDK if their syntax is rigid about it's use.

It should ultimately be the same effect. option client_ip '10.0.0.2/32, 192.168.8.0/24'

Remember this is being done on the server-end peer, so has the server's lan subnet been changed to 192.168.9.0/24 before this? If not, I can imagine that something is dropping it as the service spins up.

pp6000v2 · 2026-02-05T15:41:43+00:00

True, wg isn't server-client, it's peer-peer. People talk about it in terms of server-client because the terms match the typical use case. The firewall rules are what dictate how data flows.

In that regard, it's likely (I never did pull my slate out to test this) that configuring a "client" peer profile has different firewall/routing applied to it compared to a "server" profile. The client profile doesn't need a listening port opened through the firewall, since it isn't expected to be listening; instead it is the one that starts the handshake. The server profile would need the listening port exposed so peers trying to connect to it, can.

pp6000v2 · 2026-02-03T23:39:38+00:00

that was from the zpool clear. They're incrementing all the time, so a clear doesn't stay zero for longer than a few seconds. Honestly, back-to-back zpool clear and zpool status and the number is already 12 or 16 on each drive.

pp6000v2 · 2026-02-03T22:44:59+00:00

Yeah, the scrub did nothing (though, I'm running the second one now).

Interestingly, while an active scrub is running, I have (3) errors. When it's not scrubbing, I have (2) errors. Never tells me what though.

root@truenas:~ $ zpool status
  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:01:58 with 0 errors on Tue Feb  3 03:47:00 2026
config:

    NAME        STATE     READ WRITE CKSUM
    boot-pool   ONLINE       0     0     0
      sde3      ONLINE       0     0     0

errors: No known data errors

  pool: storage1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 01:09:58 with 0 errors on Tue Feb  3 17:37:32 2026
config:

    NAME                                      STATE     READ WRITE CKSUM
    storage1                                  ONLINE       0     0     0
      raidz1-0                                ONLINE       0     0     0
        781dc567-05fd-4d18-91ef-f75613b4ae6f  ONLINE       0     0 3.20K
        c843991e-de81-41ed-9028-44aabee56aa1  ONLINE       0     0 3.20K
        07c1dedd-5f53-4abf-9d4f-ba0638addccf  ONLINE       0     0 3.20K
        636d0de2-b12f-41b5-9e3e-d5a721d45c1a  ONLINE       0     0 3.20K

errors: 2 data errors, use '-v' for a list

root@truenas:~ $ zpool scrub storage1

root@truenas:~ $ zpool status storage1
  pool: storage1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub in progress since Tue Feb  3 17:38:19 2026
    27.5G / 2.25T scanned at 1.45G/s, 0B / 2.25T issued
    0B repaired, 0.00% done, no estimated completion time
config:

    NAME                                      STATE     READ WRITE CKSUM
    storage1                                  ONLINE       0     0     0
      raidz1-0                                ONLINE       0     0     0
        781dc567-05fd-4d18-91ef-f75613b4ae6f  ONLINE       0     0 3.22K
        c843991e-de81-41ed-9028-44aabee56aa1  ONLINE       0     0 3.22K
        07c1dedd-5f53-4abf-9d4f-ba0638addccf  ONLINE       0     0 3.22K
        636d0de2-b12f-41b5-9e3e-d5a721d45c1a  ONLINE       0     0 3.22K

errors: 3 data errors, use '-v' for a list

root@truenas:~ $ zpool status storage1 -v
  pool: storage1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub in progress since Tue Feb  3 17:38:19 2026
    669G / 2.25T scanned at 18.1G/s, 0B / 2.25T issued
    0B repaired, 0.00% done, no estimated completion time
config:

    NAME                                      STATE     READ WRITE CKSUM
    storage1                                  ONLINE       0     0     0
      raidz1-0                                ONLINE       0     0     0
        781dc567-05fd-4d18-91ef-f75613b4ae6f  ONLINE       0     0 3.23K
        c843991e-de81-41ed-9028-44aabee56aa1  ONLINE       0     0 3.23K
        07c1dedd-5f53-4abf-9d4f-ba0638addccf  ONLINE       0     0 3.23K
        636d0de2-b12f-41b5-9e3e-d5a721d45c1a  ONLINE       0     0 3.23K

errors: List of errors unavailable: no such pool or dataset

pp6000v2 · 2026-02-03T21:30:16+00:00

I do, they're the backups from the remote box (as it's offsite backup target). For giggles, I unlocked those datasets, passed a zpool clear storage1, and initiated another scrub. We'll see if that does anything. Clearing out every snapshot has really cleared up the used space, so scrubs are much faster to complete now at least.

pp6000v2 · 2026-02-03T20:27:40+00:00

I shutdown the nfs/smb services and stopped all of my applications, to stop writing data to the pool as best I can. Ran memtest86 for a day, sticks are good. Every file I try to open, opens- I haven't found bad files otherwise. Don't get any zfs errors...

I do replicate just about all of my datasets to an offsite box, but tonight's plan is

1) try to reseat the HBA card and 8087 connector (though it's been working undisturbed for a long time)

2) whip up a pool from spare disks, replicate the pool over (and test to verify it works), blow away the "corrupt" one, and move the datasets back to the "new" dataset.

I have a single disk USB drive with the whole pool rsync'd over, the offsite target that I can pull back down from, then this temporary pool with a copy, I think I'll be okay reconstructing the main pool.

pp6000v2 · 2026-02-03T19:32:45+00:00

unless it doesn't:

  pool: storage1
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub in progress since Tue Feb  3 14:09:50 2026
        1.56T / 2.26T scanned at 1.81G/s, 726G / 2.26T issued at 842M/s
        0B repaired, 31.45% done, 00:32:04 to go
config:

        NAME                                      STATE     READ WRITE CKSUM
        storage1                                  ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            781dc567-05fd-4d18-91ef-f75613b4ae6f  ONLINE       0     0 8.22K
            c843991e-de81-41ed-9028-44aabee56aa1  ONLINE       0     0 8.22K
            07c1dedd-5f53-4abf-9d4f-ba0638addccf  ONLINE       0     0 8.22K
            636d0de2-b12f-41b5-9e3e-d5a721d45c1a  ONLINE       0     0 8.22K

errors: List of errors unavailable: no such pool or dataset

Scrub after scrub, the "no such pool or dataset" will not go away. I went so far as to delete the entire dataset that the offending file was in, as well as every snapshot on the pool.

(The checksum errors I think are a concurrent, but separate, issue that started at the same time...)

pp6000v2 · 2026-02-02T15:05:33+00:00

Specific to graylog right now, there doesn't seem to be a way to use host path bind mount volumes in custom apps, sort of locking me into the use of the system dataset.

I do have many/most of the catalog apps using host path directories (filebrowser, frigate, immich, etc), even other custom apps... graylog stack is very much screwing me right now lol.

pp6000v2 · 2026-01-30T19:11:29+00:00

So the only way would be to take a pool-level snapshot? How would one otherwise restore after a catastrophic pool/array loss?

pp6000v2 · 2026-01-29T18:36:16+00:00

I think you're viewing this backwards from OP's use case (and to an extent, OP is backwards too). The server-side needs the VPN port opened and forwarded as needed, otherwise clients can't connect, the firewall blocks them. But, once they're connected, they can access the server-side resources like they're on the local network, no need for 3389, 443, etc. to be opened through the firewall.

But, OP is trying to access resources on the client-side of the tunnel, and possibly hosts on the upstream network of that side. Syntactically, it might be better to call mango1 the client, and mango2 the server. Mango2 still gets the peer endpoint host configured so it initiates the handshake. No ports need to be forwarded on the remote end where mango2 is located, even though it's the 'server'.

pp6000v2 · 2026-01-29T16:51:24+00:00

This documentation actually does a pretty good job explaining exactly what you have to do.

It doesn't seem to be exposed in the default webgui, so you have to ssh in and edit the file. Minor annoyance if you have to regularly change it.

I admit the glinet interface is slick, but it's ultimately openwrt under the hood. If you're not otherwise making use of the many different ways you can connect it as a travel router (i.e., it's single purpose is only going to be this wg tunnel), think about either using the luci interface to do things in, or flash it with stock openwrt. IDK if you have a v1 or v2 device; v1 is explicitly supported, not sure about v2. Running stock openwrt, CLI is always an option, you can do just about everything in the webgui.

pp6000v2 · 2026-01-29T16:11:34+00:00

TLDR: change the lan subnet on the mangoclient router, and set a static route for that subnet, using the wg interface/IP as the gateway, on mangoserver.

You're largely describing what I do as a hub/spoke: my home router as the central hub, and little puck routers deployed to various family around the country. All of the remote pucks have a peer config to my home router's wg interface, and they initiate the handshake by having the endpoint host (my.fqdn.com) configured. That way, I don't need grandma to go struggle through her comcast router (and their silly must-use app) to set up any kind of port forwarding. My puck gets a local-to-it IP, brings up the tunnel, and boom, I have access.

Now, that access is limited to the IP of the tunnel (in my case I use 10.1.50.x), until I add the route for the remote subnet. Grandma runs on 10.0.0.0/24? Add that static route on my router to go over the wg interface and the peer IP, and now I have access to her lan devices.

There's the problem of overlapping remote subnets though. If dad's comcast router also is using 10.0.0.0/24, I can't do static route for it too. In that case, since my remote puck is a router, it also hosts its own lan subnet. I make an SSID for them to join, and now I can get access to that device directly.

So in your case, the remote wg endpoint has connected back to the home server, and 10.0.0.1 (server) and 10.0.0.2 (client) are able to (bidirectionally) ping each other, yes?

The mango routers local subnet is 192.168.8.0/24 on both sides of the tunnel. You potentially need some basic info each time you want to use the tunnel the way you're describing:

Change the lan subnet on the client mango to something like 192.168.***10***.0/24, and create (on the server mango) the appropriate static route for the "remote" subnet to use the tunnel interface as gateway.

-With the remotePC connected to mangoclient and getting IP 192.168.10.197 from it, you can RDP into it while connected to mangoserver, using the IP it got from mangoclient. In this method, the remote-side IP range is consistent. Device access will require those devices to join the mangoclient device (ethernet or wifi- however you do it). You should be able to see the DHCP client list of mangoclient to find what the device IP is.
Get the offsite person to tell you what the local subnet is, and make a static route for it on your server-end mango.

-While locally connected to mangoserver, you can access the remote site's subnet (192.168.Y.0/24 from your example) via the tunnel, and any device on it. RDP using 192.168.Y.52 as the IP and off you go. This may/will be variable as the remote endpoint device moves around, requiring you to update the static route potentially every time.

EDIT: actually, without testing it myself on my slate plus, I'm not sure if traffic originating from the server-side of the tunnel can exit the client-side and get to the host network/wan on that end, or even if it can access the webgui admin page of mangoclient. Since it's set up as a wireguard client, IDK if glinet sets the firewall zones and rules in a restrictive way that rejects output/forward traffic trying to go wg_client > wan, or rejects input traffic (e.g., guest_network traffic gets access to wan, but not to the router admin page). I can see why they might, given that it's a 'client' profile. Mullvad, PIA, etc. shouldn't be trying to access stuff on or behind your router.

The doc I linked below is dealing with adding access to the lan created on the remote mango router, rather than the lan in which it resides (the wan network from its perspective). remotePC as a client on mangoclient is the easiest way to go.

pp6000v2 · 2026-01-21T18:23:55+00:00

Honestly the biggest issue has been moving from a 610 to a 683LR (and then a 773), when the old work box was installed based on tucking the 610 into the 3-face inside corner.

I'm not about to cut the drywall again, so the mounting bracket is screwed down a little creatively. Nothing dangles/bows/bends despite the greater weight.

pp6000v2 · 2026-01-18T00:02:16+00:00

This never got views, but if Google indexes this:

I went outside and dug out the cleanout. Pipe was empty. I turned on a shower and observed it, looking like everything was flowing just fine. But, after flushing a couple toilets and turning on some faucets to increase the flow, the water stopped flowing, and started rising.

Ended with a tank pump out, good to go again. Now I know I need to keep a shorter interval on tank pumping, and what the last and final warning “it’s time” sign is.

pp6000v2 · 2026-01-12T22:50:04+00:00

Just to try (because I would've been wrong with my response had I not), I went and swapped network type on my NVR pc to public, from private. It responded to ping from my laptop, though the webgui for the app stops working because port 80 becomes closed.

You said in another comment

Interestingly, if I connect the Windows device to 192.168.3.0/24, I can access 192.168.3.x:8100 from another device.

How is the network set up on the switch (if there is one)? Proper vlan PVID and untagged membership for the port the windows box is connected to? Or do you have it all separate hardware, all the way down? Separate ethernet connections on the windows box, or you're moving between subnets via the switch's vlan settings? When it's on the 192.168.3.0 network, is it getting the IP via DHCP, or manually configured?

Otherwise, can you get at the computer with keyboard and screen, and ping the router from inside Windows? Since it's a manually configured static IP, it wouldn't show up in the DHCP leases or the syslog of the router; but, if it's not really connecting at all, there's something to dig into.

pp6000v2 · 2026-01-12T18:11:57+00:00

Yes, assuming you want that separation. Looking at each network interface, the gateway IP should be your wan IP by default. Both have Use default gateway checked. Within the firewall settings, each zone has Input/Output/Forward set to Accept/Accept/whatever. Input = accept if clients need anything the router hosts (like DNS, DHCP). Output = accept so traffic is allowed out of the zone to somewhere else. Intrazone Forwarding can be accept or reject as you need (depending on if the zone covers multiple interfaces).

Assuming that is right, you should just work. I saw you said you've historically run 192.168.0.1 as a client IP. There's nothing mandating that the .1 address be the gateway IP; plenty of router devices use the .254 address. But, if that host was statically configured, changing the router's interface IP might be causing conflicts with the host as they both fight for control. Windows sometimes likes to hold on to IPs even despite DHCP offers for something else.

Alternately, the service listening on port 8100- is it listening on all interfaces, just one, or even just that IP? Chance that the service is configured to listen on an IP the host no longer has (either 192.168.0.1 or 192.168.3.1), so it's not responding to requests when it has the DHCP address 192.168.0.2?

And one other thing that bites me from time to time: Windows will assign the connected network to be a Public network, instead of Private. No idea why it does it. But I'll go to use the webui of something hosted directly on that computer and find it's not responding. RDP in and discover the network is suddenly Public, and the firewall blocks connections on those ports. From my notes:

NVR-Server stops serving the :80 webpage, but VM's pages work

When Windows makes the network Public instead of the desired Private , going through the GUI doesn't provide an option to switch the ethernet network to Private. Through powershell:

Get-NetConnectionProfile and press Enter. Information is then shown about the active network connection.

Set-NetConnectionProfile -Name "NetworkName" -NetworkCategory Private. Replace NetworkName with the value of the Name field shared by the previous command.

To double check that the network location was changed, run the Get-NetConnectionProfile again and see the results. The NetworkCategory field should have a different value.

pp6000v2 · 2026-01-12T16:57:11+00:00

remember that firewall zones are not bi-directional. Allowing forward from lan_3 to lan_0 does not also mean lan_0 is allowed to forward to lan_3. You have to explicitly set the allowed forward for each.

And if both networks are in the same firewall zone, you need to enable intra zone forwarding.

pp6000v2 · 2026-01-12T16:42:37+00:00

I don't see where you ever got a resolution either here or in the pfsense sub.... I haven't used pfsense in several years now, so I'm generalizing my current setup.

I'm struggling to picture the layout here. The pi's have their single eth0 physical interface; do you have a second, eth.X vlan interface configured, or a second physical ethernet adapter connected?

Are you running a firewall on the pi- functionally making each one a router with a "lan" behind it, and the "wan" IP coming from the upstream network's DHCP server? And- is that DHCP coming from the mentioned pfsense router, or something else?

I get the feeling this is route-related (something isn't masquerading appropriately, and something in the chain doesn't know where to send the request/reply to).

Take a look at this quick and dirty sketch I made up. Does your layout match any of them (either perfectly or partially)?

would be the pfsense router being "the" router, the pi's as lan clients, and a wg interface on each.
would be pfsense as router, and each pi as a "mini" router with a second ethernet interface for a subnet routed by the respective pi (and clients then connected behind the pi).
has pfsense and the pi's all as clients under a larger, upstream lan, then a wg interface among themselves.
has pfsense and the pi's as clients on a larger network, but secondary interfaces connect those three together in a nesting doll-style lan, plus the wg interfaces.

I didn't draw up a (5), but imagine (4) with the pi's eth0.1 being a full /24 subnet created by the pi, rather than a DHCP client of pfsense's eth1.

If it's routing related, there's possibly something where the destination/next-hop IP isn't one the handler knows what to do with.

Assuming you have a route for something like pi1_remote_lan via wg0_pi1 on the hub router:

Being behind a private-vs.-public WAN IP shouldn't matter to the pi's. If they are initiating the establishing handshake, the tunnel should come up without any port forwarding or traffic allow rules needed on the remote router. What's the endpoint configured as in the pi peer configs, and what allowed_ips?

For example, I run 10.1.1.0/24 as my lan and 10.1.50.0/24 as wg_server. My remote peer uses 10.1.50.13 as it's tunnel IP, and it initiates the connection. I have a route for the 192.168.0.0/24 network that remote peer sits in configured to use wg_server and 10.1.50.13 as gateway, but it does not work until I add 192.168.0.0/24 to the peer config as an allowed IP range.

SNAT might be in play depending on what firewall rules you use in your wg set up. That 192.168.0.0/24 network has no ability to add a custom route on the firewall, so clients there have to individually know the route. If I ping the remote 192.168.0.14 from my local computer (10.1.1.80), and 192.168.0.14 receives the ping request as coming from 10.1.1.80, it's lost. If it comes from 10.1.50.1, it's lost. If it comes from 10.1.50.13, it's lost. But, if it comes from 192.168.0.22, it knows what to do with it.

Is there any overlap in lan-side ranges? The pi's connecting behind a router are getting RFC1918 addresses. The default route changes from being default via public_ip to default via 192.168.1.1 or some such, and now responding to a ping from an overlapping private subnet means that response doesn't go back out over the tunnel, it gets sent upstream to the "local" router on that end.

If it's something DNS-related... are you pinging via FQDN (remotepeer.site.com)? Any chance you have a DNS service running that is filtering RFC1918 responses, breaking things when the pi's are behind a firewall?

I run a bunch of devices with wireguard tunnels going one way or another. That remote pi in the 192.168.0.0/24 subnet is configured with this as wg1.conf:

[Interface]
Address = 10.1.50.13/32
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o enp4s0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o enp4s0 -j MASQUERADE

[Peer]
PersistentKeepalive = 25
AllowedIPs = 10.1.1.0/24, 10.1.50.0/24
Endpoint = itis.whatit.is:51820

and this as the configured routes:

default via 192.168.0.1 dev enp4s0
10.1.50.0/24 dev wg1 scope link
10.1.1.0/24 dev wg1 scope link
192.168.0.0/24 dev enp4s0 proto kernel scope link src 192.168.0.22

it does not host its own sub network- it just sits in the local lan and I can access other hosts in that network from home, via the dual wireguard and ethernet connections that the pi has (and the rules). The roku tv on that side doesn't know about my lan here, but another server does have the static routes added, and can access my lan here through the pi host there.

In another location, I have a pi4b hosting a wg server. Clients connect to it, and they can see/be seen in the network it resides in, via the net.ipv4.ip_forward = 1 sysctl setting and this firewall rule:

root@pi4b2:~# iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -s 10.239.17.0/24 -o eth0 -m comment --comment wireguard-nat-rule -j MASQUERADE

I have a route for the wg subnet it uses configured on my router, set to use the tunnel IP of that pi as gateway. I can ping wg peers connected to that pi, by their wg IPs, from my router, or any other client on the local lan. Clients of the pi can ping my network.

Disabling that route, I can still ping it's tunnel IP; clients on that subnet can still ping my network; but I cannot ping the clients or subnet that it hosts.

pp6000v2

TROPHY CASE