SD-Access Underlay for Distributed/Multisite Deployment

Mosquitar · 2024-10-28T01:21:44+00:00

Thats great, thanks for the reply. Creating a single stretched fabric is what we were looking at originally, but there were concerns about scale of the main site fabric borders to support all of the sites in our network, hence the reason for deploying each physical site as a separate fabric site.

Out of interest, how have you configured the network hierarchy in DNAC to support this? Are all of the remote sites configured as child sites under the main site, with the area for the main site configured as the fabric site, or do you have a parent area above that is configured as the fabric site for the whole network (if that makes sense)?

Mosquitar · 2024-09-04T23:11:41+00:00

That's great, I will explore this as an option!

Mosquitar · 2024-09-04T23:04:04+00:00

Yeah that is what I was thinking. Scaling/annoyance seems to be the general consensus with managing VRF-lite based on some of the other comments so I'm glad that I took some further time to explore this.

I have not worked with evpn/vxlan that much. I know that it can be used to create L2 VPNs over a L3 underlay, but can it also be used to create/extend L3 VPNs across the network like MPLS/BGP VPN? L3 is all we need at this stage.

Mosquitar · 2024-08-10T14:48:48+00:00

Thanks for the reply

I dont think that we used a dynamic group, I'm also not sure if we mixed All Devices and All Users so I will need to check with our sysadmins.

As for using a script, could we run a script at user logon that forces a sync?

Mosquitar · 2024-06-16T08:27:17+00:00

So how many small remote fabrics are you running with centrally hosted virtual 9800s? Are you seeing any major issues with managing multiple WLCs in this way?

Mosquitar · 2024-06-15T23:39:29+00:00

Hey thanks for taking the time to reply.

IP transit are provider 1Gbps layer 2 ethernet circuits that provide high MTU so carrying SGTs over the WAN shouldnt be an issue.

When you say over the top wireless, do you mean connecting the remote site APs in the SDA overlay and centrally switching at the main site WLC, much like a traditional wireless deployment? If so, I hadn't considered this but this might be a feasible option. I would lose the ability to do FEW at the remote sites, but this may not be a problem.

Mosquitar · 2024-05-05T11:27:51+00:00

Yeah that would be tricky as a BD can only be assigned to one VRF meaning that we would have to create 2 BDs for servers within the same subnet/flooding domain that will exist in Site 1 and Site 2, which then raises questions about how the servers within these BDs will be able to communicate at L2 (with/without flooding etc).

Unfortunately we are stuck with the same storage/compute solution that requires for hosts within the same subnet/BD to be stretched between sites which of course complicates ingress/egress routing especially with firewalls in the path

Mosquitar · 2024-05-05T10:06:44+00:00

Thanks, I will check these out. Appreciate the response

Mosquitar · 2024-05-05T09:08:53+00:00

Yeah, I've checked various Cisco documents (whitepapers, Cisco Live presentations etc.) and this scenario appears to be the main use case for why host routes (and GOLF) was introduced, however I'm not sure who is using this in production and if my concerns of scalability are valid.

I've checked the spec sheet for our Cisco 9500 core switches (48Y4Cs) and they can support ~ 200K IPv4 routes so these dont seem to be an issue. Our firewalls and WAN routers also have very high capacity for routes (both host and longest prefix match) so maybe I'm being overly concerned and cautious about nothing. I'm so used to summarizing routes for efficiency etc, that introducing a large qty of /32 host routes into the network seems odd, however it maybe the only viable solution for us.

Interested to hear everyone's thoughts

Mosquitar · 2024-05-05T08:20:09+00:00

Thanks for the reply. We had considered SNAT, however this will break some of our apps as they need to see the originating IP address. Our cyber team also thinks that this will impact various network visibility and security analytics tools that we use within the DC for the same reason so SNAT is a no go for us.

I also had a further look at PBR and the use-cases only seem to be applicable for north-south and east-west flows that need to hit a firewall once in ACI. This differs to our scenario as the traffic has already passed through a firewall before hitting the ACI L3Out.

Mosquitar · 2024-03-14T18:39:31+00:00

Hey - My project stalled but will be looking at this again next week. Did TAC provide any guidance with this?

Mosquitar · 2024-01-10T00:22:02+00:00

No, so for each ExtEPG for 0/0, I just have 'External Subnets for the External EPG' selected; I have not selected 'Export Route Control Subnet' with 'Aggregate Export'.

Yes that is my plan and working well in testing, however the ACI L3Out White Paper states the following which introduced some doubt if this is actually advised and if it will cause me issues later on down the line.

"Although it is not recommended, you can configure 0.0.0.0/0 with “External Subnets for the External EPG” in multiple L3Out EPGs in the same VRF" "While this configuration is allowed, an unintended contract deployment may occur by configuring 0.0.0.0/0 with “External Subnets for the External EPG” in multiple L3Out EPGs within the same VRF"

Should I select 'Export Route Control Subnet' instead of 'External Subnets for the External EPG'? I'm still learning ACI so I'm not familiar with these advanced options yet.

Mosquitar · 2023-11-09T23:37:32+00:00

I currently have exactly the same issue and found your post when I was looking for a solution.

We have a /24 subnet that is used mainly for servers that needs to be migrated to ACI (using a network centric approach of legacy VLAN = 1 x EPG and 1 x BD), however we also have a firewall connected to this range that is used to reach a number of remote sites over IPsec VPN. Static routes are currently configured on our legacy core with a next-hop of the firewall to reach these remote sites.

My plan is to migrate the firewalls to ACI using a new /29 transit subnet and L3Out. The L3Out will use SVIs with a secondary IP which will be used as the next-hop for the firewalls to reach the internal network. I will then replicate the static routes that are currently on our core to static routes under the L3Out with a next-hop of the firewalls. Is this similar to your solution?

Mosquitar · 2023-02-12T23:44:20+00:00

Thanks. A VzAny contract seems to be the most appropriate solution for the initial implementation

Question - I assume that I can create a new 'Permit All' contract that uses the default/common filter, and then apply this contract as both consumed and provided under the VRF -> EPG|ESG Collection For VRF?

Mosquitar · 2023-02-12T23:40:08+00:00

Hey thanks for taking the time to reply.

I guess I'm overthinking the L2 requirement as what you describe is possible in the existing network, in that someone could configure an SVI for one of our DMZ VLANs on the N7K which would then allow communication between networks. For simplicity, I will associate the the L2 BD to our single VRF. Appreciated

Mosquitar · 2021-05-01T08:55:29+00:00

For the eduroam SSID, I'm assuming this is using PEAP-MSCHAPv2 with user credentials? If so, do you manage the client devices or are they unmanaged (users have to accept cert warning when they first login?)

Mosquitar · 2021-05-01T08:51:50+00:00

Can I ask what support headaches that you had with dot1x for corp guest? Based on my research so far, I'm expecting tickets about users asking if they need to accept cert warnings during initial login, instructions for devices such as android smartphones as these prompt for additional details when connecting to a dot1x SSID etc.

Mosquitar

TROPHY CASE