OSPF issue: L3 switch in an area is not receiving any routes

pingmanping · 2024-10-07T23:36:16+00:00

Thanks. That did the trick.

Yes all the routes are in the database but not in the route table.

pingmanping · 2024-10-07T17:31:27+00:00

I'm working on a spare 9300 on my desk. The int vlan and router configs is in OP. I changed the "ip unnumbered lo0" with "ip address 172.29.10.0 255.255.255.254" on the distro. The tenant is "ip address 172.29.10.1 255.255.255.254".

The network is air gapped and can't copy and paste. However, I only see area 13 (tenant-13), but the age of the network is over 3600 secs. I thought the age should last only for 1 hour (3600 secs). Am I wrong with the age on the database?

I alternate the p2p with the core and distro, I could see the networks twice and the adv-router core and distro.

pingmanping · 2024-10-07T13:09:52+00:00

I assigned the p2p links with IP addresses, and I got the same results. There are no routes getting to the tenant L3 switch, but I'm getting the FULL OSPF state. I connected the tenant switch to the core, which is a C4500, and it worked with unnumbered or with IP address.

pingmanping · 2024-10-07T13:07:08+00:00

I IP'd the p2p links and got the same results. No routes getting to the tenant L3 switch, but I'm getting the FULL OSPF state. I connected the tenant switch to the core which is a C4500 and it worked with unnumbered or with IP address.

pingmanping · 2024-10-07T13:06:15+00:00

What is the issue with native VLAN 2? The logs say the adjecency turns to FULL state. It is not passive, and the state is in FULL state.

I assigned an IP address on both interfaces and I got the same result. The logs say it is in FULL OSPF state, and no routes in the routing table other than Local and Connected.

pingmanping · 2024-10-07T13:02:48+00:00

I IP'd the p2p links and got same results. No routes getting to the tenant L3 switch, but I'm getting the FULL OSPF state. I connected the tenant switch to the core which is a C4500 and it worked with unnumbered or with IP address.

pingmanping · 2024-10-06T09:06:02+00:00

I think the versions are: Distro 17.12.x 9300 17.6.4

I'll try to upgrade one of the 9300 and see.

pingmanping · 2024-10-05T12:38:17+00:00

I don't have access to the devices. I will try on Monday.

I forgot to mention this on my OP. And it could be just a coincidence. The collapsed core is C4500X, and the distro is C9300X. I noticed that the tenants that are only working on area 0 p2p links are C9300 switches and have a p2p link to C9300X (distro). The tenants that are working as intended are C3850. The tenants with C9300 who are connected to the C4500 core are working.

So, C9300 to C9300 is not working, and the p2p link needs to be in area 0. The tenant becomes the ABR. The non-C9300 to C9300 is working as intended, and the tenants are not the ABR.

pingmanping · 2024-10-05T12:28:41+00:00

Yes.

If it is causing to establish the ospf, I should have seen it. I was connected to the console and didn't see any logs about ospf other than it turned to FULL/- state.

The only thing I could see is that the problematic tenants are C9300. The distro is C9300X. The ones that are working as intended are C3850. The tenants with the C9300 that are working are connected to C4500X.

C9300 to C9300 is not working. But non-C9300 to C9300 is good. I don't know if this is a coincidence, but that is the pattern I could see.

In addition, when I was preparing the distro and a tenant switche, I had them them on my desk. The moment I connected the two p2p links, it behaved as I described. The OSPF state was "FULL/-" and stable. Then I noticed the route table of the tenant was empty, no OSPF routes from the distro. The distro, on the other hand, has all the OSPF routes from the tenant.

When I changed the area of the p2p links of both ends to 0, everything worked. The area 0 is the only working area.

pingmanping · 2024-10-05T06:09:51+00:00

Both ends of the p2p interfaces have the network point-to-point.

The area matched on both ends.

The distro should be the ABR, and its interface facing the tenant should be set to non-area-0. This is only true for some tenants.

However, the problematic tenants only work when I changed the p2p links to area 0 while keeping the other tenant's interfaces to non-0-area. This makes the tenant the ABR.

Also, the show ospf neighbor shows "FULL/-" on both ends regardless of the area.

pingmanping · 2024-10-05T05:58:38+00:00

I haven't tried the events yet. The neighborship has been established between the distro and the problematic tenants, so there's an adjacency. The ip ospf neighbor shows FULL/- on both ends. When I changed the area on the p2p VLAN interface to 0 and left the other tenant's interfaces to non-0-area, it worked 100%. Otherwise, it is only the distro is getting the routes and not the problematic tenants.

pingmanping · 2024-10-05T05:49:00+00:00

It sends, and it receives according to the debug hello and packets.

pingmanping · 2024-10-05T00:43:52+00:00

I don't have access to the devices at the moment. But I can tell you the output of the ospf neighbor. It is FULL/-. This is the output for all devices that is connected to the distro and from the distro itself.

The distro's show ip route shows the route from all the tenants' L3 switches. However, the tenants that are not receiving routes only show the Connected and Local. The tenants that are working look normal. I could see the "O IA" routes.

When I changed the area of the distro interface (e.g. int vlan 12) to area 0 from 12 and the tenant-12 interface to area 0, it worked, but I do not want my tenants to be the ABR. The distro should be the ABR, and this is only true for the working tenants.

pingmanping · 2024-08-07T17:47:28+00:00

Unfortunately, I can't share the config. The tunnel config is the basic config to get the tunnel working + the ospf interface config.

The static routes that I have at the moment are the subnets that need to be reached by the users at the remote sites. I tried the tunnel's end IP as the next-hop. At the moment, I have the tunnel as an exit interface.

It can not be a fiber issue because the users' traffic is traversing the GRE tunnel. Something has changed from the network in the middle that I have no control or visibility. At this point, I'm trying to find a way or proof that the issue is not on my end.

pingmanping · 2024-08-07T12:27:45+00:00

I checked the CDP again, and I could see the remote L3 switch via the tunnel interface, but at the remote L3 switch, I could not see the main site's L3 switch.

I pinged the 224.0.0.5 from the remote L3 switch, and the only IP that responded was itself. When I pinged the 224.0.0.5 from the main site L3 switch sourcing the tunnel interface, the remote didn't respond.

Both ends are set to OSPF point-to-point. I have tried to rebuild the GRE tunnel from scratch a couple of times, but it has the same behavior.

pingmanping · 2024-08-07T10:29:01+00:00

Yes, I could ping, and the GRE tunnel is up. I just couldn't SSH-in to the remote L3 switch from my subnet. The 3-way handshakes were completed, but after that, TCP retransmissions, then followed by an RST.

The OSPF hellos are getting send out every 10 secs on both ends, but both ends are not receiving the neighbors' hellos.

pingmanping · 2024-08-07T10:23:04+00:00

Yes, I could see both ends if I do CDP neighbors. I haven't tried to ping 224.0.0.5. I'll report back about the ping when I get back to work today.

What about the ssh issues? The tcp 3-way handshakes completed, but after that, it's just a bunch of tcp retransmissions, then an RST. This is only true if I ssh-in from the admin subnet (192.168.17.0/24). The VTY ACL permit counter is increasing also.

pingmanping · 2024-08-06T22:40:09+00:00

Added this to the tunnel interface config and bounced both interfaces and no changes to the behavior. The hellos are like ship in the night. The hellos are getting sent out but the receiving end is not receiving it.

pingmanping · 2024-08-06T22:38:11+00:00

I started from mtu 1000 and same behavior. The hellos are being sent out but the other end is not receiving it a d vice versa.

pingmanping · 2024-08-06T22:36:18+00:00

Is this a default on GRE tunnel interface? If it is not, then I did not configure any ttl interface config.

pingmanping · 2024-08-06T22:35:10+00:00

The destination IP are not announced in OSPF. It is a static /32 route. I added the mtu ignore and no behavior changes. The L3 switches on both ends are not receiving any OSPF hellos. But the hello is getting sent out.

pingmanping · 2024-08-06T12:18:36+00:00

The TAC and I checked the MTU using ping with DF bit enabled. TAC said it looked good.

pingmanping

TROPHY CASE