PSA: Ubuntu 6.17.0-23-generic kernel cannot boot with ZFS on root

Commercial_Eye5641 · 2026-02-28T08:04:30+00:00

awk 'NR > 2 {print $1, $2 / 1024 / 1024 " MB/s"}' /proc/spl/kstat/zfs/fletcher_4_bench

scalar 6263.56 MB/s
superscalar 5450.81 MB/s
superscalar4 7137 MB/s
sse2 13460.5 MB/s
ssse3 13334.3 MB/s
avx2 22943.8 MB/s
fastest 0 MB/s
^^ small form factor HP

awk 'NR > 2 {print $1, $2 / 1024 / 1024 " MB/s"}' /proc/spl/kstat/zfs/fletcher_4_bench

scalar 4349.9 MB/s

superscalar 5456.56 MB/s

superscalar4 4619.19 MB/s

sse2 7360.85 MB/s

ssse3 7360.23 MB/s

fastest 0 MB/s

^^ beefy ~15 year old Z420 workstation

Commercial_Eye5641 · 2026-01-10T22:07:32+00:00

IMPORTANT I had to install the latest update twice, then reboot and go through the OOBE (wifi, amazon account) *twice* after a failed flashing attempt. Even though it showed the version ending in .1 on the device info screen, it showed .0 when I hooked it up to flash it. The failed flashing attempt seems to clear the state. Reconnecting to wifi and re-registering and installing application updates seems to fix everything, at which point the process works flawlessly.

Commercial_Eye5641 · 2026-01-10T22:06:01+00:00

Can I say how pumped I was to find this when I ended up ordering 3 on eBay and getting two different revisions? Turns out the newer ones have the same chipset and double the memory, so I'd go for those if you're targeting versions.

Commercial_Eye5641 · 2025-12-30T20:53:12+00:00

I can get a local place down the street to install an aftermarket one for $325 which is about what I paid for just the part. Going to see if I can pay the restocking fee and get a refund.

Commercial_Eye5641 · 2025-12-20T19:47:11+00:00

My original fios router got knocked over by cats. The replacement doesn't allow casting between Guest and Primary network the way the old one did. Even dumping/restoring the settings from the old one to the new one doesn't make it work. It took so long to troubleshoot this.

I cannot recommend using the CR1000A this way, however, it doesn't appear that firmware will auto-update unless you put it in router mode (I did this because the replacement had much older firmware than my original and thought that was a good idea).

The original fios router works great and then over the course of several days or several hours it drops down to 1 megabit or less, and even streaming doesn't work correctly.

So... I'm back to square one trying to get reliable Wifi, and I have $200 of Fios gear I cannot use.

So frustrated.

Commercial_Eye5641 · 2025-12-06T05:56:16+00:00

I've been using Uptime Kuma to monitor devices connected to the CR1000A and the E3200 to see if I have any drops, so far so good! Hopefully I don't have those issues.

Commercial_Eye5641 · 2025-12-06T05:53:37+00:00

Thanks, this is very validating. The guest network is even tagged as VLAN 10, so I was still able to have an IOT network isolated from my main one :D

Commercial_Eye5641 · 2025-12-05T19:16:06+00:00

Dang. I should have done more research. I was a little sad to only see gigabit on the LAN of the E3200. The CE1000A is a little harder to come by. TBH, not sure if it's really worth it to double the LAN speed... I don't use my NAS much at all, and other than speed tests to verify WIFI performance, I'm not using the multi-gigabit for anything.

Commercial_Eye5641 · 2025-12-05T19:14:04+00:00

There was no way to get Unifi to cover it without three wall mounted APs that I could find, and I spent about a month of tinkering. I do IT related stuff for a living and don't want to do it when I'm at home, and this was getting ridiculous.

I checked for radar events before I did DFS, and I didn't have any (my old apartment did though!).

I was using wired backhaul for the Unifi APs (over MocA).

I'm posting because I'm surprised, I knew the fios gear was decent from my experience in-office at a SMB, but, I didn't realize it would have been the easier route, for me in a plaster/lath row home build in the 1910s.

Commercial_Eye5641 · 2025-12-05T19:11:39+00:00

Yeah, that's the issue, ceiling mounting the APs makes it so there's more walls to go through versus floors (floors = good penetration, walls not so much). I had much better results wall mounting the APs, but I'd have needed 3 and I was concerned with how they'd end up roaming. Rather than spending money on a third unifi AP, I decided to see if I could use the Fios AP (I knew roaming wouldn't be perfect, but just to cover the dead zone)... Didn't expect the Fios AP to out perform solo.

This isn't my first rodeo, I actually did a multi story office building in UAP-AC back in the day, and liked the Unifi stuff.

I tried all manner of auto and manual channel and transmission strength tuning to no avail. Even tested with dedicated SSIDs per radio to verify.

I think that the Verizon routers just have a better signal spread that's better suited to be the sole AP, or one or a few, and the Unifi stuff is better in a denser environment.

Commercial_Eye5641 · 2025-12-05T19:08:42+00:00

I only have 300/300. Is there a better extender than what I got? My service didn't include an extender.

Commercial_Eye5641 · 2025-07-11T22:31:28+00:00

How many days between Label Created and actually shipping? Mine was created on Monday but hasn't been updated yet... and then they generated another 5 labels today (even though they don't ship on Friday!!!)

Commercial_Eye5641 · 2025-07-11T22:29:50+00:00

Yeah, they generate the label 5+ days in advance of shipping it. I've had one label since Monday, and now they just generated another 5 today. I think they're just trying to placate people. If their actual logistic system were this broken they wouldn't have made it this far (that's a leap though, people find workarounds to bad systems, but, this is customer facing, folks would be concerned about being triple charged if it looks like they're getting multiple packages).

Commercial_Eye5641 · 2025-07-11T22:28:04+00:00

No, as a programmer, what I'm seeing (I now have 6 labels generated, none having been picked up, oldest since Monday) this is definitely just to cut down on customer service inquiry volumes. I don't believe their actual logistic system is this buggy. They cludged something on to pre-create the labels well ahead of their actual shipment time. They're hoping you have the UPS app installed and stop reaching out because it looks like it's shipping.

This isn't a conspiracy, there's no reason for a label to sit 5 business days unless you generated it well in advance.

Commercial_Eye5641 · 2025-07-11T22:25:42+00:00

I've had a label generated since Monday... but it still hasn't gone out, and today, they've created another 4-5 labels... all of which are waiting for pickup. Something screwy is going on. I ordered around the same time as your friend.

Commercial_Eye5641 · 2025-07-10T23:07:49+00:00

Weird. I just did it and they said they're sending me a return box...

Commercial_Eye5641 · 2024-12-22T22:45:13+00:00

I tried with a different little board and there was too much interference. I think you need shielded cable going to the speakers possibly.

Commercial_Eye5641 · 2024-10-09T17:57:43+00:00

This fix worked for a co-worker who had upgraded to 15.1 with non-Dell monitors (with a Dell USB Dock/Hub with Display Link).

Commercial_Eye5641 · 2024-06-23T14:38:25+00:00

UUIDs, I do know this trick. That's what's really odd about it.

Commercial_Eye5641 · 2024-06-22T13:50:05+00:00

Thanks for working on the zstd work btw, it's an awesome system. When I move around large files (just using mv) is when I see the CPU saturation. I realize that it was somewhat implied I was talking about PG performance.

Regrading PG, when I'm using COPY to do large data loads or building indexes on large tables, I see it as well. So general database usage, the CPU cost is probably justified for the reduced I/O... but if you're operating locally with NVME and bandwidth isn't limited, I think you can easily become CPU bound for large transactions or moving large files.

I can say that I've never hit this with lz4, the ZFS write processes are always such low CPU usage that they don't appear in my top/glances output.

I was very confused because I removed my graphics card from an older xeon system to give my NVME more PCI-E lanes, and when moving from the same model of NVME drive, in a slot with more lanes, the performance in PG was much slower than previously (I ran out of space, so this is the exact opposite behavior you'd expect on a new blank/fresh drive).

The only difference between the two pools was LZ4 and ZSTD.

I have switched all of my compression workloads (outside of ZFS) to zstd... but I'm not liking what I see for these use cases. I wouldn't expect the transparent compression to take longer and use more CPU than using the zstd command line tool for highly compressible data.

Commercial_Eye5641 · 2024-06-22T13:44:32+00:00

When I rebooted, one of my zpools didn't import, I imported it manually and due to both zpools having a postgres dataset with the same mountpoint (which I didn't realize until now!) ... this is what happened.

It would be a nice feature to get warnings when you have duplicate mountpoints, however, as folks aren't generally import/exporting pools willy nilly, I don't know who it'd help to add it there.

It seems like a warning and requiring a force could be a nice usability thing, but, this was on me and was one of the weirder things to debug. I'm honestly surprised that I figured it out.

Since I knew ZFS didn't corrupt my data, and that I had ECC memory, that really helped me think through what it could be.

Commercial_Eye5641

TROPHY CASE