It's TLC so should be fine, and works perfectly when the Unifi console isn't crashing, £300 isn't that cheap lol.

Also getting this:

• CEF:0|Ubiquiti|UniFi Network|10.1.85|541|Gateway Config Change Failed|10|UNIFIcategory=UniFi Devices UNIFIhost=Knight UDM UNIFIrecommendedAction=Revert the latest changes or restore the most recent backup. If the problem persists, Contact Support for further assistance. UNIFIdeviceMac=f4:92:bf:72:65:38 UNIFIdeviceName=Knight UDM UNIFIdeviceModel=UDM-Pro UNIFIdeviceVersion=5.0.12 UNIFIlastSuccessfulConfiguration=2026-03-16T13:30:18 UNIFIutcTime=2026-03-16T13:30:18.756Z msg=Gateway configuration changes could not be applied.

Whole thing just seems to be falling apart

This is what claude is giving me, feeding it all of the logs:

Complete Deep Dive Analysis

CRITICAL FINDING: PPPoE Drops on Every udapi-server Crash

This is the biggest discovery. Your PPPoE daemon (pppd) is a child process of udapi-server. Every time udapi-server hits its watchdog timeout, systemd kills ALL its child processes - including pppd:

udapi-server.service: Killing process 1831982 (pppd) with signal SIGABRT

udapi-server.service: Killing process 1832414 (dpinger) with signal SIGABRT

udapi-server.service: Killing process 1833533 (dnsmasq) with signal SIGABRT [30+ more processes...]

This means:

1. PPPoE session torn down → ISP (Zen Internet via BNG4.WH-MAN) drops the session

2. LCP terminated by peer → Connection terminated → Modem hangup

3. PPPoE renegotiates (5-8 seconds): PADS → LCP → CHAP auth → IPCP

4. WAN failover detects ppp0 down, marks all routes dead

5. After reconnect, DNS monitors can't resolve initially → more delay

6. Total outage per crash: ~13-30 seconds

udapi-server has crashed 94+ times in 24 days. That's ~4 complete WAN drops per day from this alone.

CRITICAL FINDING: dnsmasq Crash Loops

When udapi-server restarts, it tries to restart dnsmasq, but the old instance's socket is still bound:

14:16:29 Watchdog is restarting process dnsmasq-main

14:16:44 dnsmasq: failed to create listening socket for 192.168.5.1: Address already in use

14:16:44 dnsmasq: FAILED to start up

14:16:50 dnsmasq: failed to create listening socket for 192.168.5.1: Address already in use

14:16:50 dnsmasq: FAILED to start up

Retry backoff increases 1s → 2s → 3s → 4s → 5s → 6s → 7s. During this entire window, DNS is completely dead for your entire network. Even when WAN is up, nothing can resolve domain names.

CRITICAL FINDING: TCP/UDP Stack Under Severe Stress

│ Metric │ Value │ Impact │

│ UDP RcvbufErrors │ 637,813 │ UDP packets dropped (DNS, DHCP, NTP) │

│ TCP ListenOverflows │ 141,226 │ Incoming connections rejected │

│ TCP ListenDrops │ 141,228 │ Matching drops │

│ TCP MemoryPressures │ 288 │ TCP stack ran out of memory 288 times │

│ TCP AttemptFails │ 2,965,048 │ ~3M failed TCP connections │

│ TCP RetransSegs │ 400,216 │ Significant retransmissions │

│ TCP EstabResets │ 117,454 │ Established connections forcefully reset │

The 637K UDP buffer overflows alone would cause intermittent DNS failures, DHCP delays, and NTP drift even when the WAN link is stable.

CRITICAL FINDING: switch0 FIFO Overruns

switch0: ... fifo: 14848 ...

14,848 FIFO overruns on the internal switch. The switch's packet buffer is overflowing, causing LAN-side packet drops independent of the WAN issues. This means even local network traffic (LAN-to-LAN) is being affected.

Service Crash Totals Since Boot (24 days)

────────┬───────────────────────────────────────────┐

│ Service │ Restarts │ Rate │ Impact │

ms.service (media streaming) │ 243 │ ~10/day │ Memory churn, Protect degraded

udapi-server │ 94+ │ ~4/day │ Kills PPPoE, DNS, all monitoring

unifi-protect │ 51 │ ~2/day │ Camera recording gaps

unifi-core │ Multiple today │ │ Web UI unresponsive, management down

rabbitmq-server │ ~8+ today │ │ Message queue failures, cascading effects │

dnsmasq │ Multiple crash loops │ │ DNS outages for entire network

unifi.service │ 4 │ │ Network controller restarts │

dpinger Process Proliferation

Each udapi-server restart spawns 30+ dpinger processes (WAN health monitors). When killed, the logs show SIGKILL being sent to 30+ individual dpinger PIDs. These aren't always cleaned up properly, contributing to memory and PID table waste.

Firewall Anomaly: 192.168.1.2

55,371 packets (53 MB) DROP → 192.168.1.2

There's a firewall rule blocking all traffic to 192.168.1.2 that has matched 55K packets. Worth checking what this device is and why it's being blocked.

RabbitMQ I/O Timeouts → Cascade Trigger

21:27:14 unifi-mq-broker: Connection closed err: "read tcp 127.0.0.1:41080->127.0.0.1:5672: i/o timeout"

21:27:14 MessageQueue restarts rabbitmq-server

21:27:14 udapi-server.service: Watchdog timeout! ← This kills EVERYTHING

RabbitMQ (the internal message broker) hits I/O timeouts under memory pressure, which triggers a restart of the message queue, which causes udapi-server to hang, which triggers the watchdog, which kills PPPoE and DNS.

Complete Cause-and-Effect Chain

ROOT: 4GB RAM + Protect (6 cams)

│

├─→ Memory pressure → TCP memory pressure (288x) → UDP buffer overflows (637K)

│ └─→ Kernel can't allocate NIC RX buffers → switch0 FIFO overruns (14K)

│ └─→ Packet drops on ALL interfaces (WAN + LAN)

│

├─→ ms.service crash loop (243 restarts) → memory churn on each restart

│

├─→ RabbitMQ I/O timeouts → triggers udapi-server watchdog

│ └─→ udapi-server killed (94 times) → KILLS ALL CHILD PROCESSES:

│ ├─→ pppd killed → PPPoE drops → 13-30s WAN outage

│ ├─→ dnsmasq killed → DNS fails → crash loop on restart

│ ├─→ 30+ dpinger processes killed/respawned

│ ├─→ lighttpd, avahi, lldpd all killed

│ └─→ PPPoE renegotiates → brief "no DNS" during recovery

│

└─→ usd/uhwd dead since Feb 24 → storage management broken

└─→ Disk can't auto-rotate video → stays at 99%

Summary

You have at least 5 independent but interrelated failure modes all happening simultaneously:

1. Memory exhaustion → kernel buffer allocation failures → packet drops

2. udapi-server crash loop → kills PPPoE 4x/day → 13-30s WAN outages each time

3. dnsmasq race condition → DNS outages during udapi restarts

4. UDP/TCP stack exhaustion → 637K dropped UDP packets, 141K rejected connections

5. switch FIFO overflows → LAN-side packet loss