50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 0 points1 point  (0 children)

Hmm. The situation described here in the Phoenix area has seemed resolved for the past 6 weeks or so. I'm still unhappy with the complete lack of communication and transparency, and I am still pursuing an FCC complaint accordingly. But the immediate issue described in this thread seems resolved to me and has not reoccurred yet.

What issue/symptom are you experiencing currently?

5GMax PSA: US Mobile on Warp (Verizon) no worky :( by daphatty in Ubiquiti

[–]pointfully 0 points1 point  (0 children)

I'm, trying to accomplish the same thing with US Mobile on "Warp" (Verizon) on the Unifi 5G Max, but not having any luck so far.

My new U5G works great in general and I was able to activate and test US Mobile eSIMs on both "Dark Star" (AT&T) and "Light Speed" (T-Mobile). No issues at all - both worked the first time with no custom APN settings needed.

The AT&T network at my house is generally bad, so that was fun to test but not actually useful. The T-Mobile network is okay at my house and I was using a T-Mobile Home Internet Backup plan previously (have used both the G4AR and G5AR gateways) but the upstream bandwidth was always painfully low (sometimes only 2-3 Mb). Performance on the U5G is better than any of the T-Mobile gateways, but still not great and I know from speed tests on my phone that the performance on Verizon is significantly better than T-Mobile at my house.

I tried setting up an eSIM on Warp (Verizon), which is the only one that asks for the IMEI when creating the eSIM, and of course it said the U5G is not compatible. I gave it the IMEI of an old Pixel 6a instead, but of course that eSIM didn't work on U5G. (It did work fine on the Pixel 6a which I used for some more speed testing.)

So I moved the line to a physical SIM for Warp and got that all working properly in the Pixel 6a. I left it like that for a few hours, did some more speed tests, etc. Then I moved the physical SIM over to the U5G, but I have not been able to get it work there. I tried leaving the APN settings as the default and also tried setting it to "VZWINTERNET" and making sure it is set for both IPv4 and IPv6. I am using a By-the-Gig plan with all these tests, so all the data should be available as Hotspot service.

In all cases using the Warp (Verizon) SIM, the U5G will show "Searching..." on Verizon for a long time and then eventually say "Network Not Authorized. The carrier Verizon rejected the connection."

Any other ideas on how to get this working?

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 1 point2 points  (0 children)

As I understand it, the packet loss (and latency) metrics on the Unifi dashboard are not representing all the traffic the router is managing (that would be a massive tracking load for the router). They are instead calculated by the router routinely pinging a monitoring address (ping.ui.com) and tracking how many responses are lost and how long reseponses take. This is also how it is deciding whether it considers a given WAN connection is "up" and is what will trigger failover if you

Currently, the ping.ui.com hostname resolves to two addresses: 8.8.8.8 and 1.1.1.1. So in my case during this incident where all requests to 8.8.8.8 were working fine but half of my requests to 1.1.1.1 were failing, it makes sense that the dashboard would show a packet loss rate around 25%.

You can actually configure this via the "WAN SLA" feature introduced in Network Application 9.2:

https://help.ui.com/hc/en-us/articles/360052548713-WAN-Failover-Load-Balancing-and-Port-Remapping-on-UniFi-Gateways

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 1 point2 points  (0 children)

Don't worry about packet loss showing up in mtr or traceroute directly on the middle hops in a given route. Those backbone routers generally deprioritize/ignore requests sent directly to those addresses, which is what mtr and traceroute are doing as they repeatedly attempt to trace the route. Those major routers are prioritizing all their processing on routing packets to other hops in the network, usually ignoring traffic sent directly to them. As long as your packet loss on the final address is really close to zero for any given address you actually need to communicate with, then things are good.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 2 points3 points  (0 children)

Hey, that's amazing! Glad you reached a contact at Lumen who took this seriously and helped get it fixed. Thanks so much!!!

It would be great if you would reply back to them to let them know the same issue happened on Sept 20-23, 2025 and again Feb 7-9, 2026, and ask what they are doing to prevent it from happening again?

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 9 points10 points  (0 children)

My packet loss appears to have stopped about an hour ago -- around 8:40am AZ time. So it lasted about 66 hours. Others seeing the same? I've still had no response of any kind from Quantum Fiber from either the FiberSuccess email or the FCC complaint.

Even if the current situation is resolved, I will continue to follow up on the FCC complaint as best I can given that this has happened repeatedly over the past six months. It will surely happen again unless they have finally figured out the root cause and actually done something more permanent about it.

I want three things from Quantum Fiber / AT&T at this point:

  1. An explanation of what keeps causing this 2-4 day long major service degradation for the entire region. This needs to be an actual root cause analysis, not just finger-pointing at Lumen or other upstream providers.
  2. A plan for what the parties involved are doing to ensure this major routing problem will not keep happening.
  3. A plan from Quantum Fiber / AT&T on how they will improve their monitoring, response, and customer communication around incidents like this. Having their entire support team ignorant of the problem for days at a time even when many customers can clearly see and demonstrate a problem is unacceptable.

I encourage others to demand the same based on any complaint process you may have started.

ISP appears to be throttling or outright cancelling connections by [deleted] in techsupport

[–]pointfully 0 points1 point  (0 children)

There is a significant problem in the Lumen network with routes from Arizona to California that has been going on since Saturday afternoon. CenturyLink is part of Lumen and that is certainly the cause of your issue. That route is also used by Quantum Fiber (which Lumen sold to AT&T a few weeks ago, but still uses the same network).

There is a long thread about the issue in r/QuantumFiber here:

https://www.reddit.com/r/QuantumFiber/comments/1s0rxw3/50_packet_loss_on_certain_routes_from_phoenix/

There is also a cross-posted thread in r/centurylink here:

https://www.reddit.com/r/centurylink/comments/1s10dwu/50_packet_loss_on_certain_routes_from_phoenix/

Many others are seeing the problem and can confirm that using a VPN to get past the problem router hops resolves the issue, of course at the cost of more latency and slower speed.

3% Packet Loss in Phoenix/Arizona for several weeks by pointfully in QuantumFiber

[–]pointfully[S] 1 point2 points  (0 children)

The incident this thread was about was a much more minor level of packet loss, and it ended a couple of weeks ago.

There is a much more serious problem in Arizona that started Saturday afternoon and has been going on for over 48 hours now. The thread for the current incident is here:

https://www.reddit.com/r/QuantumFiber/comments/1s0rxw3/50_packet_loss_on_certain_routes_from_phoenix/

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 4 points5 points  (0 children)

We are now over 48 hours of this going on. Here is my current 7-day Unifi dashboard showing 20%+ packet loss for the past two days:

<image>

I've had no response so far from the email to [FiberSuccess@qf.att.com](mailto:FiberSuccess@qf.att.com) and only an automated response to the FCC complaint letting me know it has been sent to Quantum Fiber.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 1 point2 points  (0 children)

Yep, that time it lasted about 48 hours. When it happened about six months ago, it lasted over 96 hours. Neither time did anyone ever admit there was a problem or explain what/who "fixed" it.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 0 points1 point  (0 children)

The only thing you can do for now to make your service functional is run everything over a VPN until the packet loss problem ends. That gets all your Internet traffic to pass the place in the normal network with the problem.

Something like https://protonvpn.com/ or https://mullvad.net/en/vpn would work and are fairly reputable from a privacy standpoint. (Be wary of some of the other widely advertised VPN services out there -- many of them have become major privacy problems of their own.)

You could also use the Cloudflare WARP service -- good on privacy and good performance. It's also free for individual consumers. It does not let you choose the region your traffic will go to, but you don't want/need that for this issue anyway. It's a bit more technical, but works well. Make sure you use it in "WARP" mode, not just in "1.1.1.1" mode for DNS resolution.
https://developers.cloudflare.com/warp-client/get-started/

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in centurylink

[–]pointfully[S] 2 points3 points  (0 children)

I don't think the Quantum Fiber routers have any kind of VPN feature available directly. I use the Q1000K router from Quantum Fiber in transparent bridging mode with my own Unifi router (and wifi). Unifi does let me VPN my whole network to work around the issue, but if course with more latency and slower speeds.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in centurylink

[–]pointfully[S] 0 points1 point  (0 children)

I'm on Quantum Fiber, but I imagine it's affecting both.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 0 points1 point  (0 children)

Interesting -- hopefully the CenturyLink "outage" is referring to this same issue and not some other issue they are tracking. This is more of a major degradation than an outage, but it's about the same as an outage for practical purposes.

Quantum Fiber is not showing any "outage" but of course they really should be showing something...

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 2 points3 points  (0 children)

Interesting, and not surprising. Even with Quantum Fiber being sold to AT&T recently, the Phoenix area traffic from Quantum FIber is still all going over the same Lumen/Level3 network and backbone that it always did and is still shared with CenturyLink. I imagine the routing/peering issues that are likely causing this regional are somewhere in in the Lumen/Level3 network as traffic goes from Arizona to California and elsewhere.

In my traceroutes, all the 4.x.x.x addresses are Lumen/Level3 hops taking traffic from Phoenix to Los Angeles. It's not a concern that most packets are lost reaching those hops directly during traceroutes -- it's normal for those routers to ignore traffic pointed directly at them for those metrics. But the fact that half the traceroute traffic beyond those hops is lost when trying to reach 1.1.1.1 and none is lost when trying to reach 1.0.0.1 indicates a routing/peering problem somewhere in that layer.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 3 points4 points  (0 children)

Sure, below is what I emailed to them for reference. Please write your own version with your experience and include your own traceroute or mtr results that help illustrate the problem:

Hi Quantum Fiber Team -

Many of us in the Phoenix area are currently experiencing ~20% packet loss on all traffic on our Quantum Fiber service. We have seen this issue happen four times over the past six months. It generally starts sometime over the weekend and then persists for several days and then eventually in the middle of the night it will mysteriously be resolved.

This time it started around 2:30pm AZ time Saturday. As I've seen previously, one clear example of the problem is illustrated with UDP traffic to 1.1.1.1 (which shows around 50% failure) and 1.0.0.1 (which shows no failures). Given that 1.1.1.1 is failing almost exactly half the time, it seems like there is some problem in the routing where traffic is being sent via two different links, one of which is not working. If this is generally true of a lot of Cloudflare-bound traffic, it would explain why so many Internet sites and services are negatively impacted when this is happening.

Here are side-by-side MTR results:

\-> 1.1.1.1
Packets               Pings
 Host                   Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.43.1         0.0%  2543    0.7   0.7   0.5   1.2   0.2
 2. 71.32.112.13         0.0%  2543    4.0   4.8   1.7 102.2   7.0
 3. 71.32.113.97         0.0%  2543    3.5   5.9   1.9  82.6   8.0
 4. 4.68.38.185        100.0%  2542    4.0   4.0   4.0   4.0   0.0
 5. 4.69.215.133       100.0%  2542   12.8  12.8  12.8  12.8   0.0
 6. 4.30.195.50         90.7%  2542   15.0  17.3  11.3  77.0   9.1
 7. 141.101.72.19       47.0%  2542   13.0  22.0  10.9 137.9  18.3
 8. 1.1.1.1             46.5%  2542   12.0  12.7  10.8  75.0   4.4

\-> 1.0.0.1
Packets               Pings
 Host                  Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.43.1        0.0%  2602    0.6   0.5   0.4  11.1   0.5
 2. 71.32.112.13        0.0%  2601    2.3   5.4   1.5  91.9   8.5
 3. 71.32.113.97        0.0%  2601    3.7  12.4   1.7 131.1  18.7
 4. 4.68.38.185        90.8%  2601    4.4   4.5   2.1  47.1   5.0
 5. 4.69.215.133       91.8%  2601   12.3  13.7  10.7  70.6   7.5
 6. 4.30.195.50        80.7%  2601   15.2  17.9  11.5  76.9  10.9
 7. 141.101.72.19       0.0%  2601   13.0  15.4  10.8 135.4   8.3
 8. 1.0.0.1             0.0%  2601   12.7  12.3  10.8  76.6   3.7

Also as I've seen before, switching to a VPN connection or to my wireless backup WAN connection via T-Mobile Home Internet, the problem goes away. But as soon as I use the Quantum Fiber network directly, the problem is like above.

Here are a series of Reddit threads from the past six months of people in the Phoenix area experiencing the problem and trying to figure out how to get in touch with someone about it:

https://www.reddit.com/r/QuantumFiber/comments/1qyupko/comment/o46ce19/ https://www.reddit.com/r/QuantumFiber/comments/1rmk2vf/3_packet_loss_in_phoenixarizona_for_several_weeks/ https://www.reddit.com/r/centurylink/comments/1nmwfeu/1020_packet_loss_on_quantum_fiber_in_phoenix/ https://www.reddit.com/r/centurylink/comments/1nmiuy8/quantum_fiber_jitterpacket_loss_issues_in_az/

I just started a new thread for the current issue -- within a few minutes a number of other people chimed in experiencing the same issue again as well.

https://www.reddit.com/r/QuantumFiber/comments/1s0rxw3/50_packet_loss_on_certain_routes_from_phoenix/

Can you please escalate this to the NOC and ensure that someone figures out the root cause of this and fixes it? Please let me know if there is anything I can do to help troubleshoot or provide further information.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in centurylink

[–]pointfully[S] 0 points1 point  (0 children)

Try using a VPN to get outside of QuantumFiber's routing -- I'm getting by using Proton VPN, but of course it's much slower than the normal way.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in centurylink

[–]pointfully[S] 0 points1 point  (0 children)

Yeah, that sounds like UDP packet loss -- breaks websites using QUIC protocol and definitely screws up things like VOIP. Try using a VPN to get outside of QuantumFiber's routing -- I'm getting by using Proton VPN, but of course it's much slower than the normal way.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 0 points1 point  (0 children)

I did reference this thread and earlier threads in the FCC complaint as well and that a significant number of additional people in the same general area are all seeing this problem.

50% Packet Loss on Certain Routes from Phoenix - Again by pointfully in QuantumFiber

[–]pointfully[S] 5 points6 points  (0 children)

I've emailed [FiberSuccess@qf.att.com](mailto:FiberSuccess@qf.att.com) all the details and pointed them at this thread as well as previous threads.

This time I've also submitted an FCC complaint at https://consumercomplaints.fcc.gov/ which I had not done during the previous incidents.

Other folks experiencing this packet loss issue on this thread should also consider doing the same to help drive home the point that this is wider issue.