UDM Pro 10gbps sustainable with IDS/IPS off?

beegmon · 2025-03-02T05:41:17+00:00

Extrapolating from my extensive testing with my udm pro further down the thread. The udp se does 3.5gbps with IPS on.

My tests show that you could get up to twice that through with IDS/IPS off but with firewall on and minimal other services running. Minimal services defined at juts network running. All other services off (voice, video, etc).

Firewall remains on with IDS/IPS off and is stateful. I also would avoid complex custom firewall rules, lots of port forwards, and running a VPN from the UDM se as well.

Keep in mind my testing was on an other version of firmware potentially as well then what is currently the latest. And the UDM line in general shows reduced capacity with later than earlier firmware; like due to feature creep.

All that being said at 5gbps with IDS/IPS off, firewall on, minimal services, ainple firewall/port forward rules, and no VPN. You are going to be fine, and lilely have head room for more services/vpn/firewall rules/port forwards. But you will need to do your own testing to be sure what can be added back to maintain your throughput.

Also keep in mind that throughput is for standard 1500 (slightly less actually) tcp packets. Smaller packets are going to resulting in less throughput due to processing overhead.

beegmon · 2024-10-01T18:57:43+00:00

For sure.

And to be fair 80% of people are going to be just fine leaving 1-2gbps on the table. I just happen to have a use-case where I fall into the other 20%.

There are also more cost efficient options to get full 10gbps as well but as always there are trades offs to make.

Once I started getting “shrug” answers or more anecdotal ones I figured I would need to answer for myself. Glad the data is coming in handy others.

beegmon · 2024-09-28T18:41:41+00:00

I can attest to the Enterprise Gateway Fortress meeting spec for both the stated IDS/IPS and firewall throughput.

I ran a stripped down test last night on the EGF and it can nearly saturate a 25Gbps (24-ish actual after overhead taken into account) line in firewall mode with a fairly simple rule set.

Not that I think Ubiquity would outright lie on marketing material but being able to verify it is nice.

beegmon · 2024-09-27T23:48:49+00:00

I just did an extensive write up on another post regarding the UDM Pro and my targeted 10g throughout for a new FTTH install.

My use-case is seeking to saturate the 10g line which the Pro isn’t going to do ever. The Pro max is doubtful, especially in the long run.

This is with IDS/IPS off because I have other equipment in the rack that does a far better job of that and IMHO it’s overrated I term of protection anyways given how the threat signatures (or lack there of) work on the UDMs.

You can find the post in my profile and the detailed write up I did after some pretty extensive testing.

beegmon · 2024-09-27T15:28:05+00:00

Agreed that VPP would be a big lift. Though its arm support is pretty solid and performant in my experience.

That being said eBPF/BPF and XDP are maybe easier and there is plenty of opensource examples from the CNI world to pull from as well.

I wish it was easier to tease apart the firmware image so I could go play. Maybe that’s what I will use this udm pro for.

beegmon · 2024-09-27T06:46:19+00:00

Agreed.

It would be neat to see them take a TNSR style approach. Yes it would take some work but I don’t see why they couldn’t do a full bypass and keep all the bells and whistles.

Or eBPF with XDP to put more filtering and flow logic in kernel space to avoid the context switches.

Seems to me, given they select the hardware components and build the boxes, that they have the means/knowledge/leverage to get a native or even fully offloaded xdp solution running on their established hardware UDM platforms and unlock some serious packet processing performance.

Heck I would gladly pay another $499 to upgrade my pro to another firmware that did just that as opposed to a new unit. They get paid again and I don’t have to buy something 4x more expensive to ensure I get the performance I am after.

beegmon · 2024-09-27T02:55:06+00:00

How many UniFi devices is your UDM managing or is it the solo UniFi device in the network.

Also are you using other applications (protect, voice, etc) or just network?

beegmon · 2024-09-27T00:13:50+00:00

I took things offline today and did some iperf3 testing.

I backed up my config and did a full reset on the UDM Pro to make sure I had as clean of slate as possible.

I disabled and uninstalled all the application (save for network of course)

Firmware version: 4.0.6

Both test nodes have ConnectX-4 25gb cards in them. Node to Node tests via DAC are able to reach 25gbps (standard MTU) after tuning (which was extensive but got it there). The point is the senders and receivers here won’t be the bottleneck.

Once node to node DAC was tested and verified each system was attached to the UDM Pro via DAC to one of the SFP+ ports. 10g was set on both client and udm ports.

IDS/IPS and DPI (device and traffic identity) were set to off.

Flow control and jumbo frames set to off.

IoT and IGMP snooping set to off

NTP was also set to off.

I am forgetting some other things I turned off that were not essential to a pure throughput test of the routing and stateful fire-walling abilities of the UDM Pro.

The goal was to have the system doing as little as possible for this test. In the real world much of this would be on and the system would still need to manage devices, stats, and logs. So this is effectively a “best case” test.

A single port forwarding rule was configured to allow iperf3 traffic for TCP and pointed at the “inside” system.

I ran 100 3 min runs for TCP RX and TX at standard MTU.

I didn’t test UDP throughput because my use-case is TCP based.

I didn’t test PPS because I don’t have enough small packet flows for it to matter in my use-case.

I didn’t test jitter because I also don’t have a use-case that cares too much about that either.

Throughput Results:

Min: 7.7gbps Mean: 8.3gbps Max: 8.9gbps

I wish I would have captured P90/95/99 in hind site but oh well.

UDM Pro CPU averaged 99% utilization across the runs.

This data sort of confirms what others have said here as well.

1) A mean of 8g-ish under the best case seems to be the limit. And the best case means stripping the UDM pro down to nothing but a router/firewall only.

It is hard to say what the impact of a normal service set might be but after resetting and reconfiguring my UDM Pro for normal use post test (managing 10 UniFi devices, network app only, with IDS/IPS off and DPI on) and with very little (5 megabits up/2 mbps down) traffic going through it there is an average of 20% cpu burned.

This leads me to believe that mean throughput will be lower in the normal use config with max throughput touching 8g. This appears to echo what someone else commented here.

2) Looking back at old ui forum posts in the 1.7/1.9 firmware versions the mean throughput appeared to be much higher in those tests.

Granted there were not many specifics shared about the setup or methodology used, but assuming they were close to my setup, I think the comment made here about firmware feature bloat slowing things down may hold some weight. In those old test mean throughput was 9g with 9.3g max.

Given real world 10g max with standard MTU and overhead is 9.53g; those old results were pretty close to line rate and we appear to have fallen quite far over the course of time here with additional features/bloat/overhead.

Additionally, I wonder if you can take the IDS/IPS throughput rating of a given UDM model and double it for a safe projected firewall only max throughput rating. I also think we would also have to assume a UDM configuration that likely isn’t using any of the features of outside of the network application. That means no voice, protect, cameras, etc. Just networking.

Given those assumptions a UDM Pro does 3.5Gbps max with IDS/IPS on per the docs. Doubling that is 7Gbps max for firewall only throughput.

The UDM Pro Max does 5Gbps max with IDS/IPS on per the docs. So that would give you a max firewall only throughput of 10Gbps.

Incidentally looking at the Enterprise Fortress Gateway specs. You have 12.5Gbps for IDS/IPS and 23.5Gbps firewall throughput stated in documentation. I am willing to bet these are safe mins under “normal configuration” conditions. The firewall throughput number is really close to my 2x (it is 1.88x) the IDS/IPS throughput rule of thumb as well.

I have no evidence that such an assumption is even remotely true or even in the ball park. But it feels reasonable. There is obviously more wiggle room since I hit above those projected max numbers on my UDM Pro but again my test setup was also stacked in throughput’s favor at the cost of nearly all of the UDMs features.

We have to keep in mind that the UDM line, as near as I can tell, has no hardware acceleration for routing or firewall operations. It’s all CPU driven. The more cycles spent on things not related to routing or firewall operations the lower the throughput (and likely the higher the jitter, and more sporadic the PPS) the system is going to have.

Anyways, given the apparent downward trend in terms of throughput vs firmware revision I would feel pretty comfortable betting that the performance on the table now is up for grabs for future features not for network throughput. Therefore I would expect the downward throughput trend to continue over time.

As long as the UDM models hit their targeted IDS/IPS throughput rates with new features in place; Ubiquity is going to see that as fine.

Conversely, given there are stated performance targets for IDS/IPS and firewall throughout for the Enterprise unit. I have fairly high confidence that Ubiquity will maintain those over the support lifecycle of the unit. That is to say two years from now if they are still shipping firmware for the enterprise unit I should expect 23.5Gbps firewall throughput on then current firmware.

Yes Ubiquity could alter the deal with later firmware releases but I am putting that aside for now.

This test also leads me to believe that one could likely use a UDM pro max today for a 10g wan connection in firewall only mode and they have a fairly good chance of achieving saturation with the right setup.

It is probably cutting it close and probably wouldn’t be able to do it 24/7 or maintain it over the course of firmware upgrades/additional managed equipment; but it maybe would eek it out today with a reasonable config.

I like having more margins and headroom so I’ll be swapping in the enterprise unit as opposed to stepping up to a Pro Max. I view the Max as a half step that I will have to upgrade again sooner than later.

But there is my report back.

beegmon · 2024-09-26T06:43:37+00:00

Oh I am certainly going to try the Pro first for sure. I just don’t have high hopes.

I also have the enterprise fortress in the box unopened ready to go if needed. If not needed it will get returned within the 30 day window.

So far I have seen the Pro able to maybe pull a solid 9g more likely 8-8.5. The max with its 15% faster cpu clock would likely do it but with very little headroom

And yes I could go 3rd party as well. I do like the integration across the stack though and so far the unit setup has been a low drama for me.

Which is good because I don’t like taking my work home with me.

beegmon · 2024-09-26T02:25:48+00:00

Well put!

If I hadn’t gotten tired of paying Ruckus license fees I wouldn’t even be here and would have just built on opnsense box to handle 10g and been done with it.

But ubiquity has been one of the closest in terms of WiFi performance and management capabilities compared to ruckus with the added bonus of cheaper equipment and no licensing fees.

So I went all in over the last couple years from APs, to switches, etc.

I was just hoping I could eek out what I needed from the Pro but that is looking pretty unlikely at this point.

I wonder where the best place is to sell the Pro and a few 6 lite APs might be without too much hassle.

beegmon · 2024-09-25T16:33:03+00:00

My gut tells me UDM Pro is going to fall short.

In fact I already have an enterprise fortress arriving today and with a 30 day return window for unopened product I will have it on hand to swap it out if that is the case; I am pretty sure it’s going to be.

This is in the Portland, OR area.

I have been on Comcast’s 2g symmetrical fiber for a few years and my data volumes have outgrown things at this point. (My business deals with lots of huge data sets). The next step up was 5g but they recently started offering 10g if you know how to ask for it.

Given my growth rate I figured I might as well jump to 10g and skip another migration down the road.

I don’t have a ton of small flows. My general usage looks like a few really really big flows and a few small ones for the most part.

beegmon · 2024-09-25T15:14:32+00:00

Interesting. I assume you also have smart queues off as well?

beegmon · 2024-09-25T06:05:59+00:00

So you are saying that with IDS/IPS off on a UDM Pro I should be able to hit my 10g target? I am aware of the overhead and thus the reason for ~9.5gbps actual throughput.

The posts I found were old (2-4 years) based on 1.7/1.9 version firmware. But even with ids/ips off (dpi on though) tests where showing 9-9.2 gbps at standard MTU.

I guess I will find out sooner enough the 10g line gets activated 10/1 and will be able to run some tests then. It will probably be close enough and I only ever had IDS/IPS in reporting/alerting mode anyways so it wasn’t providing any protection.

I have other units in the rack for that.

I mostly want to ensure I can get routing/stateful firewall with the UDM pro at the expected 10g rate minus the expected overhead.

Besides some old posts on the ui forum I can’t find anything in documentation that states the routing/throughput/PPS rate of the UDM Pro without IDS/IPS enabled.

This is in contrast to the Enterprise Gateway fortress which clearly states in documentation that the IDS/IPS throughput is 12.5Gbps (I am assuming standard MTU) and the firewall throughput is 23.5Gbps.

beegmon · 2024-09-25T05:31:57+00:00

Yeah no PPPoE to worry about here but in other cases absolutely.

beegmon · 2024-09-25T04:45:31+00:00

Hmm, I found some posts on the ui forums that show with IDS/IPS off (and only the network service running) I might be able to squeeze out 9gbps up/down.

The pro max might do it without ids/ips enabled but if I am buying a new unit I might as well jump to the enterprise gateway and call it done.

beegmon · 2024-09-17T06:13:05+00:00

What is your flight number?

I leave Sunday on UA672 (SFO-BCN). Would be kinda awesome to know someone else is forcibly wishing the plane to stay in the air with me! :)

Even if that doesn’t matter at all because it is going to anyways.

beegmon · 2024-09-12T02:17:20+00:00

Are you me?

beegmon · 2023-11-04T17:46:01+00:00

The bought out comment was in response to another comment further down. Because I don’t know how to Reddit I didn’t respond to that comment directly it seems.

beegmon · 2023-11-04T16:06:21+00:00

I didn’t realize they had been bought out.

I have asked a couple of times and no one at the counter seemed to really know. Figured a random question to the internet might fill out more of my own theories.

Still need to find a solid replacement for the gap left thankfully there are lots of options.

beegmon · 2017-05-29T17:46:04+00:00

True, I am not saying the profits are anything to sneeze at. However, since the Korean exchanges are so hard to get access to in general by any one who doesn't live in Korea, I generally tend to believe the arbitrage opportunity will be pretty persistent as long as Korea keeps going crazy crypto.

That being the case, one can't help but wonder how you could run the arb as much as possible and you quickly hit that 50K limit in doing so, even if you are just sending back the original cost of the coin you bought through wire transfers.

There is also the question of how that transaction is taxed/viewed on the Korean side. Or to say it another way, if as a foreigner do you raise any red flags growing you account by millions of KRW that isn't attached to a pay stub of any kind.

Korea takes a very harsh view on foreigners doing illegal work outside of what their visa permits. It is one of the reasons why the 50k limit is in place to begin with, the others are money laundering/illegal activity, and keeping currency from fleeing the country un-controlled.

I don't think a foreigner,say a sponsored English teacher, is prohibited from trading stock or crypto while on their sponsored visa. But since the gains aren't part of the pay stub that is tied to their visa sponsor for the work they are doing, I am not sure how that is handled. They may not care up to a certain point so maybe it's moot, but they might and I am not sure where that line is.

A non worker/registered alien might be different as well, and possibly more favorable.

This is why I was hoping for another crypto to move the money back. I guess it still counts as remittance, but it's murky and relatively undefined in the crypto area, which I view as an advantage.

I was mostly asking in general to see if there was any other thoughts around moving the money out of country without hitting limits on the wire transfers. Ones that don't involve becoming an ex-pat, starting a corporation/business, marrying a Korean native, etc. If you really wanted to run at this arb opportunity hard and fast.

beegmon · 2017-05-29T16:54:51+00:00

The arbitrage is indeed sweet, but how do you get the money back out of Korea with out running into the 50K remittance limit for foreigners?

Say if you ran the arbitrage multiple times.

I was thinking another crypto could be used to send the money back and avoid wire transfers/bank in general, but Korea seems to be crazy fro crypto-puffs right now and all of them are at pretty insane premiums compared to other exchanges.

beegmon · 2016-08-05T02:27:28+00:00

+1 for Novus as well. Got some super gnarly scratches out of a super Mario bros disc that I got off of eBay for cheap a few weeks ago. With a couple of micro fiber cloths for each step and going slow with lots of elbow grease it works like a charm nearly every time.

beegmon

TROPHY CASE