IPv6 forwarding issue with OSPFv3 in a user VR

[Paul_Thornton]

Hi all

Before I open a TAC case about this, I'm just trying to find out if anyone else is doing something similar and has the same issue.

Somewhat simplifying, I have four devices physically connected like this:

[ Cisco 1 ] -------- [ X460G2 "RX1" ]
: :
: :
: :
: :
: :
[ Cisco 2 ] -------- [ X460G2 "RX2" ]
On each physical link, there is a point-to-point VLAN in a user VR, 'VR-Internet', with v4 and v6 addresses. This appears to be the important point.

These all run OSPF, OSPFv3 and iBGP between them. Happy little routing network 🙂

Except that recently we had a failure of Cisco 2 - and I noticed that the IPv6 iBGP sessions between RX2 and RX1, and between RX2 and Cisco 1 went down.

Some troubleshooting later and it seems that the following case is true if all links are up:
1) It all works as expected.
2) You can ping RX1's loopback from RX2, v4 and v6.
3) OSPFv3 shows adjacencies all up. iBGP all up.
4) On RX2, the IPv6 next-hop for RX1's loopback is via the link directly to RX1 as expected.

If you break the link between RX2 and Cisco 2 (or Cisco 2 goes away):
1) Things don't work as expected.
2) You can ping RX1's loopback (and everything else in the network) from RX2 on IPv4.
3) OSPFv3 still shows adjacencies are up between RX1 and RX2.
4) On RX2, the IPv6 next-hop for RX1's loopback remains the same, the directly connected link to RX1 as expected.
5) However, you cannot ping RX1's loopback from RX2 on IPv6.
6) Nor can you ping Cisco 1's loopback from RX2 on IPv6.
7) Unsurprisingly, given (5) and (6) above, iBGP goes down.
 If I work in VR-Default on these switches, and configure some IPv6 between RX1 and RX2, things work as expected. This is a key point, it seems that the forwarding inside the VR is the problem and not OSPFv3 in general.

I'm in the process of trying to reproduce in the lab on a much simpler config (in theory, this can be done with two X460 G2s with two VRFs and one point to point link).

Has anyone else seen anything similar? I'm running 21.1.3.7 on these X460s.

Paul.

[Jarek]

When you do show route, do you see the 'f' ? (provided to fib)
Example:
#oa 172.16.20.0/24 172.12.2.21 1 UG-D---um--f- 526d:1h:52m:59s

Maybe you can change the IP's and paste the 'show iproute' from the devices ?

--
Jarek

[Paul_Thornton]

I think the issue may be a RIB -> FIB problem.

If I ping across the directly connected link, it works fine. The moment I go 'one hop' further (eg: to the loopback of the switch) it fails.

Both devices have sensible routes to each other - using the directly connected link's addresses (well, it is v6 - so the link local addr is the next hop).

The part that really confuses me is that in this broken state, if you reconnect the Ciscos, it becomes possible to ping RX1's loopback from RX2 again. With no change of best next hop or anything else I can see from the output of 'rtlook xxx'. Very, very odd.

Paul.

[Jarek]

Are the routes provided to FIB ?
Or maybe only visible in RIB ?

Can you for example ping IP from RX2 to RX1 ? ( I mean the directly connected IP's )

--
Jarek

[Paul_Thornton]

Hi Jarek,

Absolutely. The route to RX2's loopback is present and correct in all cases, so OSPFv3 is doing the right thing. It is almost like I've forgotten to 'enable ipforwarding ipv6' on RX1 (which I haven't - it works - using the same route - if the two Ciscos are up and connected).

Paul.

Edit to add: I meant the route to RX1's loopback in the first line. Typo!

[Jarek]

Hi Paul,

what do you see after 'sh iproute vr VR-Internet' on RX2 and RX1?
Do you see any route to RX1's loopback ?

--
Jarek