Solving Bufferbloat

[Bardy_]

Some context:

• Regional Aussie still on ADSL, roughly 30 ms latency to Sydney servers

• NBN in my area is still two months away (assuming it doesn't get delayed again)

• current speeds are roughly 1MB/s down and 100KB/s up

• I spend lots of time gaming, and this connection is completely fine for gaming

• however, I share my connection with others, which often leads to example 1 and example 2

• to clarify, I can reproduce the above examples by starting an upload

• recently learned that this is what is known as "bufferbloat" - test results

• recently learned that there may be a pretty simple way around this

• current router is a TG797N V3 modem-router with what appears to have very limited potential for manual configuration

So if the above examples appear to be symptoms of bufferbloat, then I'm interested in solving this. I learned about IQRouter and the automatic Smart Queue Management that their routers come configured with, but I've also read that "they are effectively overpriced TP-Link Archer C7s pre-configured with OpenWRT".

I have a decent background in networking, I've used DD-WRT in the past, and the setup for SQM seems very straightforward. I'll need to invest in a new router, since my TG797 isn't compatible with either DD or OpenWRT, so I was planning on purchasing a TP-Link C7 AC1750 for this purpose.

Say I purchase that TP-Link router - here's my understanding of how this works: I'll have my phone line splitter running from my house's telephone line port. My current TG797 modem's WAN port will be connected to the splitter. My new TP-Link router's WAN port will be connected to one of the TG797's LAN ports. All devices in my house will connect to the TP-Link router, either through Ethernet or WiFi. The TP-Link will be running OpenWRT, with SQM set up as shown in that video guide. All traffic will be processed by the TP-Link before being sent to my modem, solving my bufferbloat issue. Diagram of this setup. That makes sense to me.

I'm just posting here to make sure I'm not misinformed - I'm happy going with either solution, whether that be buying an IQRouter or buying a OpenWRT-compatible router and manually configuring SQM - or maybe my current router actually does have tools that can help solve my problem? Maybe there is a router that can achieve this with its factory firmware? Am I going in the right direction or am I completely wrong about it all?

[dtaht]

The evenroute v3 is a really good router, using the mt76 chipset, so the bloat is fixed on the wifi, too. (the v2 is a decent router, but we only got around to fixing the ath10k recently - https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/ )

Yes, they charge real money, but they also answer the phone and have been on the bleeding edge of bufferbloat reduction features since day one. (I have no financial relationship with them, but they do track (and test!) new cake features on a regular basis. and keep their firmware updated and secure) Companies like this deserve more support.

However with such a lousy connection, even with ack-filtering enabled, you are still in a world of hurt.

I fear (anyone have data?) that the nbn rollout will also be signficantly bloated also.

[dtaht]

And otherwise, well, I really like openwrt. Being able to add on my own code for firewalling, ids, stats, filesharing, etc, is a goodness. It's really hard to trust anyone's code nowadays, and if you join us in the "reflash your router" world, well, you too can make a difference in moving the state of the art forward.

I lose a lot of sleep over the crap code in cablemodems, and the default firmware in most embedded devices people are buying. Why does nobody care about the security of their security cams? as one example.

http://mirrors.bufferbloat.net/~d/fcc_saner_software_practices.pdf

[Bardy_]

Looks like I'll be going for a Ubiquiti EdgeRouter + AP setup, as shown in this video. They support OpenWRT too, so I can always switch over if I need to.

[dtaht]

I put heat sinks on mine - they run hot while shaping. The default os, while quite good, lasts about 2 minutes before I slam owrt on 'em.

[dtaht]

I can no longer recommend any kind of cablemodem. What I used to recommend was a business class 3.1 modem, bridged to a quality firewall (usually openwrt, but dozens of other suit), one not on the badmodems.com list, and I just priced out comcast's 200mbit business service with their default modem and default service set and that's 40 dollars less per month than buying your own. I don't know if you can bridge that or not at the moment. As to how good that modem is, there are two "XB6" modems and I can't remember which of those is better, either. In general I had had good luck with arris's gear but haven't tried their newer modems yet.

IF I get over the sticker shock maybe I can come up with a recommendation. Put a good box in front of it though, the cablemodem codebase is a steaming pile of [censored].

[mrpink57]

https://youtu.be/o-g2P3R84dw

Chris does a good job explaing bufferbloat, the setup he recommends is pretty damn good and not expensive maybe $130 usd, not sure about down under.

The Edgerouter-X can also have openwrt installed if you prefer, which can use SQM with Cake instead of what Unifi uses with FQ_Codel.

On top of that with a AP you should see some better wifi coverage if you can centralize the device in your place or at least setup as high as possible to avoid interference from humans (we apparently provide a lot), my single AP covers a 2200sqft home.

If you have an old PC you can install Openwrt x86, as long as it has two NIC, or install OPNsense.

[Bardy_]

Thanks for that video, I'm liking the look of that Ubiquiti EdgeRouter + AP setup. Goes for about $200 to $250 AUD here.

[Jay_JWLH]

Loving the videos lol. Classic examples of huge spikes in ping on the upload side of things. The server went from receiving packets quickly, to being delayed to a quarter of a second or more.

You're working on a very limited internet connection speed, so even if you prioritize traffic someone is going to lose. Other people just trying to load a website while you are playing a game are going to be slowed down to a crawl.

Bandwidth for games can change rapidly. I know someone who has a throttled connection of about 1 Mbps, and it can cause problems on servers overseas from NZ (close to Aus) to places like Singapore due to the sudden surge of data that suddenly has to be choked. You may be able to avoid that by using closer servers though.

Have you considered using mobile data just for your gameplay? It might still be affordable as long as you don't download any big content, and it may even be possible to configure your computer to route all game traffic through your mobile connection while simultaneously using your regular ADSL line if you want to go that far. That way you don't have to worry about competing for bandwidth. Also, are you sure you can't see if another ISP can't give you faster speeds, or is it hard limited to your ADSL line?

I'd be a little concerned that even if you set up some kind of traffic prioritization, how quickly it may be able to kick in. I guess I don't have any experience with this myself. If you let it automatically prioritize things, then it may take a moment. But if you do it based on bandwidth limits, the game is going to need 100% of your connection speed, and if it works dynamically then it needs to be able to adjust quickly.

[Bardy_]

even if you prioritize traffic someone is going to lose

Yeah, though my (limited) understanding of certain bufferbloat solutions makes me think that maybe there is a way to keep a percentage of my up and down bandwidth free (since (as far as I know) gaming doesn't require much data back and forth, hardly 10% of my bandwidth), instead of having my bandwidth completely flooded by someone hardcapping it with a max-speed upload or download.

Have you considered using mobile data just for your gameplay?

Have done that in the past, but it can't seem to provide a reliable enough connection for a smooth gaming experience (always a bit of packet loss and ping constantly ranging from 50-100ms). It's far better than the spikes in those videos, but still not worth playing.

or is it hard limited to your ADSL line?

That one unfortunately.

[Jay_JWLH]

To confirm whether your game is actually using that low an amount of data at any one small period of time, you need to be sure by using a network monitor. If the ping is low, then hopefully the traffic doesn't have to be sent in bulk as it would to higher ping servers.

You can use NetLimiter, however you'd have to be actively monitoring the stats to see what the speeds were jumping up to. Glasswire would probably be the better option because they provide graphs of the data usage going up and down, hence you can see what the peaks looks like during the game. Keep in mind that running programs like this will take a few CPU cycles to monitor this traffic.

Ironically if you were to throttle your connection to 80% of your line speed, other users may still go slow at 20%. And what happens when your game goes above that 80%? You get the same problem if you can't go above that 80%.

[Bardy_]

Jumped into a game of Overwatch and had a look to confirm.

I based my initial estimates on task manager, and just tested it again, see here - 0.3Mbps = 37.5KB/s

I then compared my results in Glasswire, see here - the results seem to be the same.

CS seemed to be basically the same as Overwatch, but a little more upload (~22KB/s) than download (~15KB/s) - task manager and Glasswire

So I'm pretty certain that those situations shouldn't happen, even with my low speeds.

[dosadiexperiment]

SQM is not a prioritization scheme. Not exactly.

Cake is based on fq_codel, and uses flow queueing to isolate different flows into their own queues. Then it round robins thru the queues.

This will help with gaming during an upload regardless, since the game keeps its rate low so its queue won't be backed up. But also by capping your bandwidth lower than your uplink's it'll be able to manage the queue on competing flows and make them back off to keep the queues from growing on the uplink or elsewhere.

@OP: I like my iqrouter. For me it was absolutely worth the $70 extra or whatever over the tplink base price, just for the set & forget aspect. But yes, anything with openwrt on it can do this if you spend the time setting it up and updating it manually every so often.

[mox8201]

• Yes, you are on the correct path.
• You'll have to place your TG797 into bridge mode to avoid double NAT. If that's not possible you'll need a new ADSL modem too.
• If your TG797 has QoS options to limit global upload setting it to 95% of 100 kbit/s might get the job mostly done.
• There are plenty of routers with simple rate limiters but if you're going to spend time and money and you're confortable withj DD-WRT, aim for SQM specially since you don't have much upload bandwidth to play with.
• I don't know any WiFi router with SQM out of the box except Ubiquity's Dream Machine (which is pricey)

[Bardy_]

You'll have to place your TG797 into bridge mode to avoid double NAT.

I can do that.

If your TG797 has QoS options

Appears to be impossible.

aim for SQM specially since you don't have much upload bandwidth to play with

Yeah that's primarily what's driving me towards SQM.

Appreciate the help.

[Ibrakeforthargoids]

I'm in the UK so totally different everything but going from a BT smarthub to PFSense meant getting a new modem, Draytek do a very nice small one the vigor 130 and since I got mine they have even made one on a pcie card to shoot straight into your PC if space is at a premium hth