Traceroute through MPLS cloud

[LEM]

Traceroute through MPLS cloud.I have a simple demo stand for MP-BG. Here is how it looks:
CE1-PE1-PE2-CE2

We have BGP peering between CE and PE routers and OSPF, MPLS, MP-BGP inside provider cloud.

Both CE routers successfuly exhange routes. CE1 can see CE2 routes and vice versa.

I try to make trace from CE1 (192.168.101./0 ) to CE2 (192.168.102.0/24). Here is an output:

SW1.8 # traceroute 192.168.102.9 from 192.168.101.1
traceroute to 192.168.102.9, 30 hops max
1 172.16.13.3 1 ms 4 ms 13 ms
2 * * *
3 172.16.49.9 1 ms * 10 ms

Why do we have this string in output?
2 * * *

Here is a quoute from user guide:
In this mode, the MPLS TTL is independent of the IP TTL. The MPLS TTL is set to 255 on all packets
originated by the switch and on all packets that enter a pseudowire.

So, we should not see mpls path in trace.

[LEM]

Also, i found something i can't uderstand. SW3 SW3.9 # sh bgp routes vpnv4 all Routes: Destination Label LPref Weight MED Peer Next-Hop AS-Path ------------------------------------------------------------------------------------ * i 65102:2::172.16.49.0/24 0x00074 100 1 0 10.1.1.4 10.1.1.4 * i 65102:2::192.168.102.0/24 0x00074 100 1 0 10.1.1.4 10.1.1.4 65102 Next hop to 192.168.102.0/24 goes via 10.1.1.4 SW3.10 # rtlookup 10.1.1.4 Ori Destination LSP EndPt Mtr Flags MPLS Label VLAN #mp 10.1.1.4/32 10.1.1.4/32 1 U--D---um-Lf- 0x00434 ISL-34 I ithink that #mp means that mpls should be used. But if look up route from vpnv4-vrf in SW * SW3.11 # * SW3.11 # vr "CUSTOMER1-SITE1" * (vr CUSTOMER1-SITE1) SW3.12 # sh iproute Ori Destination Gateway Mtr Flags VLAN Duration #d 172.16.13.0/24 172.16.13.3 1 U------um--f- ISL-13 0d:1h:1m:44s #bi 172.16.49.0/24 10.100.34.4 0 UG-D---um-lf3 ISL-34 0d:1h:0m:0s #be 192.168.101.0/24 172.16.13.1 1 UG-D---um--f- ISL-13 0d:1h:0m:18s #bi 192.168.102.0/24 10.100.34.4 1 UG-D---um-lf3 ISL-34 0d:1h:0m:0s Next hop to 192.168.102.0/24 is 10.100.34.4. It is also SW4 interface (SW4=PE2) It is strange that next-hop is different. Also, if i check global vrf path route to 10.100.34.4 there is no #mp flags: * SW3.14 # rtlookup 10.100.34.4 Ori Destination Gateway Mtr Flags VLAN Duration #d 10.100.34.0/24 10.100.34.3 1 U------um--f- ISL-34 0d:1h:5m:4s May be it can be a root of problem?

[LEM]

Here is an ouput: * (vr CUSTOMER1-SITE2) SW4.83 # rtlookup 172.16.13.1 Ori Destination LSP EndPt Mtr Flags MPLS Label VLAN #bi 172.16.13.0/24 10.1.1.3/32 0 UG-D---um-lf3 0x00434 ISL-34 Traceroute path is same, because there is only 1 path there  * SW1.25 # ping 192.168.102.9 from 192.168.101.1 with record-route Ping(ICMP) 192.168.102.9: 4 packets, 8 data bytes, interval 1 second(s). 16 bytes from 192.168.102.9: icmp_seq=0 ttl=253 time=2.193 ms RR: 192.168.101.1 172.16.13.3 172.16.49.4 192.168.102.9 172.16.49.9 172.16.49.4 172.16.13.3 192.168.101.1 16 bytes from 192.168.102.9: icmp_seq=1 ttl=253 time=11 ms (same route) 16 bytes from 192.168.102.9: icmp_seq=2 ttl=253 time=12 ms (same route) 16 bytes from 192.168.102.9: icmp_seq=3 ttl=253 time=9.748 ms (same route) --- 192.168.102.9 ping statistics --- 4 packets transmitted, 4 packets received, 0% loss round-trip min/avg/max = 2/8/12 ms

[Prashanth_KG]

Hi,

Thank you for sharing the outputs.
On the customer1-site2 switch, can you please collect the output of

"rtlookup 172.16.13.1"

This will display the actual route being currently used by this switch for the destination?

Do we have the same traceroute output with both the commands below?

- traceroute 192.168.102.9
- traceroute 192.168.102.9 from 192.168.101.1

If there is a difference, please also collect the below output:
ping 192.168.102.9 from 192.168.101.1 with record-route Thanks in advance!

[LEM]

Here is a ping from CE1:
* SW1.21 # ping 192.168.102.9 with record-route
Ping(ICMP) 192.168.102.9: 4 packets, 8 data bytes, interval 1 second(s).
16 bytes from 192.168.102.9: icmp_seq=0 ttl=253 time=8.635 ms
RR: 172.16.13.1 (CE1 interface)
172.16.13.3 (PE1-CE1 interface)
172.16.49.4 (PE2-CE2 interface)
192.168.102.9 (CE2 users interface)
172.16.49.9 (PE2-CE2 interface)
172.16.49.4
172.16.13.3
172.16.13.1

It looks like that 172.16.49.4 doesn't have route to 172.16.13.1. But look at iproute table:
* (vr CUSTOMER1-SITE2) SW4.83 # sh iproute
Ori Destination Gateway Mtr Flags VLAN Duration
#bi 172.16.13.0/24 10.100.34.3 0 UG-D---um-lf3 ISL-34 0d:0h:17m:43s
#d 172.16.49.0/24 172.16.49.4 1 U------um--f- ISL-49 0d:0h:19m:5s
#bi 192.168.101.0/24 10.100.34.3 1 UG-D---um-lf3 ISL-34 0d:0h:17m:43s
#be 192.168.102.0/24 172.16.49.9 0 UG-D---um--f- ISL-49 0d:0h:18m:22s

[Prashanth_KG]

Hi,

* represents no response from the specific hop.

I checked in the lab and even if we use the MPLS for the L3 routing, those hops are displayed.

Since we are doing a traceroute from a specific source-address, does all the routers in the path knows how to reach back to this source, 192.168.101.1?
I am suspecting there could be an asymmetric routing in the path.

Please try to do a ping to the destination from the source-address with a record-route.
ping 192.168.102.9 from 192.168.101.1 with record-route More details about the record-route can be fetched from the below article.
https://gtacknowledge.extremenetworks.com/articles/Q_A/How-is-PING-with-record-route-option-better-than-tracerouteHope this helps to narrow down the reason for the reported behaviour!