topic Fabric Extend - cannot get adjacency over IP Tunnel in ExtremeSwitching (VSP/Fabric Engine)

Fabric Extend - cannot get adjacency over IP Tunnel

Erol_Asim — Sat, 16 Jun 2018 14:34:00 GMT

Unsuccessfully cannot get an adjacency in GNS3 VOSS 7.0 using Fabric Extend over IP Tunnel using loopbacks. Can anyone advise where I’m going wrong here? I think my logic is correct, but maybe the config is not? I need to implement this for a customer, I thought I would try this in a lab scenario to test the feature, but not getting anywhere with it.

Here's my topology:

On SST:
#
# ISIS CONFIGURATION
#

router isis
sys-name "SST"
ip-source-address 10.1.1.1
ip-tunnel-source-address 10.1.1.1
is-type l1
system-id 82bb.8401.0101
manual-area 49.0000.0000.0001
exit
router isis enable

#
# LOGICAL ISIS CONFIGURATION
#

logical-intf isis 1 dest-ip 10.2.2.2
isis
isis spbm 1
isis enable
exit

SST:1#show isis

====================================================================================================
ISIS General Info
====================================================================================================
AdminState : enabled
RouterType : Level 1
System ID : 82bb.8401.0101
Max LSP Gen Interval : 900
Metric : wide
Overload-on-startup : 20
Overload : false
Csnp Interval : 10
PSNP Interval : 2
Rxmt LSP Interval : 5
spf-delay : 100
Router Name : SST
ip source-address : 10.1.1.1
ipv6 source-address :
ip tunnel source-address : 10.1.1.1
Tunnel vrf :
ip tunnel mtu :
Num of Interfaces : 1
Num of Area Addresses : 1
inband-mgmt-ip :
backbone : disabled
Dynamically Learned Area : 00.0000.0000
FAN Member : No

SST:1#show ip route
=====================================================================================================
IP Route - GlobalRouter
=====================================================================================================
NH INTER
DST MASK NEXT VRF/ISID COST FACE PROT AGE TYPE PRF
-----------------------------------------------------------------------------------------------------
10.1.1.1 255.255.255.255 10.1.1.1 - 1 0 LOC 0 DB 0
10.2.2.2 255.255.255.255 10.10.10.2 GlobalRouter 20 10 OSPF 0 IB 20
10.10.10.0 255.255.255.0 10.10.10.1 - 1 10 LOC 0 DB 0

3 out of 3 Total Num of Route Entries, 3 Total Num of Dest Networks displayed.
--------------------------------------------------------------------------------------------------
TYPE Legend:
I=Indirect Route, D=Direct Route, A=Alternative Route, B=Best Route, E=Ecmp Route,
U=Unresolved Route, N=Not in HW, F=Replaced by FTN, V=IPVPN Route, S=SPBM Route
PROTOCOL Legend:
v=Inter-VRF route redistributed
SST:1#

SST:1#ping 10.2.2.2
10.2.2.2 is alive

SST:1#show isis logical-interface

========================================================================================================================
ISIS Logical Interfaces
========================================================================================================================
IFIDX NAME ENCAP L2_INFO TUNNEL L3_TUNNEL_NEXT_HOP_INFO
TYPE PORT/MLT VIDS(PRIMARY) DEST-IP PORT/MLT VLAN VRF
------------------------------------------------------------------------------------------------------------------------
1 -- IP -- -- 10.2.2.2 Port1/1 10 GlobalRouter

------------------------------------------------------------------------------------------------------------------------
1 out of 1 Total Num of Logical ISIS interfaces
------------------------------------------------------------------------------------------------------------------------
SST:1#

SST:1#show isis inter

====================================================================================================
ISIS Interfaces
====================================================================================================
IFIDX TYPE LEVEL OP-STATE ADM-STATE ADJ UP-ADJ SPBM-L1-METRIC
----------------------------------------------------------------------------------------------------
10.2.2.2 pt-pt Level 1 UP UP 0 0 20000

--------------------------------------------------------------------------------
1 out of 1 Total Num of ISIS interfaces
--------------------------------------------------------------------------------

SST:1#

SST:1#show isis int-l1-cntl-pkts
====================================================================================================
ISIS L1 Control Packet counters
====================================================================================================
IFIDX DIRECTION HELLO LSP CSNP PSNP

----------------------------------------------------------------------------------------------------
10.2.2.2 Transmitted 413 0 0 0
10.2.2.2 Received 0 0 0 0

SST:1#

#####################################################################

On WMH:
#
# ISIS CONFIGURATION
#

router isis
sys-name "WM-01"
ip-source-address 10.2.2.2
ip-tunnel-source-address 10.2.2.2
is-type l1
system-id 82bb.8402.0101
manual-area 49.0000.0000.0001
exit
router isis enable

#
# LOGICAL ISIS CONFIGURATION
#

logical-intf isis 1 dest-ip 10.1.1.1
isis
isis spbm 1
isis enable
exit

WM-01:1#show isis

====================================================================================================
ISIS General Info
====================================================================================================
AdminState : enabled
RouterType : Level 1
System ID : 82bb.8402.0101
Max LSP Gen Interval : 900
Metric : wide
Overload-on-startup : 20
Overload : false
Csnp Interval : 10
PSNP Interval : 2
Rxmt LSP Interval : 5
spf-delay : 100
Router Name : WM-01
ip source-address : 10.2.2.2
ipv6 source-address :
ip tunnel source-address : 10.2.2.2
Tunnel vrf :
ip tunnel mtu :
Num of Interfaces : 1
Num of Area Addresses : 1
inband-mgmt-ip :
backbone : disabled
Dynamically Learned Area : 00.0000.0000
FAN Member : No

WM-01:1#

WM-01:1#show ip route
=====================================================================================================
IP Route - GlobalRouter
=====================================================================================================
NH INTER
DST MASK NEXT VRF/ISID COST FACE PROT AGE TYPE PRF
-----------------------------------------------------------------------------------------------------
10.1.1.1 255.255.255.255 10.10.10.1 GlobalRouter 20 10 OSPF 0 IB 20
10.2.2.2 255.255.255.255 10.2.2.2 - 1 0 LOC 0 DB 0
10.10.10.0 255.255.255.0 10.10.10.2 - 1 10 LOC 0 DB 0

3 out of 3 Total Num of Route Entries, 3 Total Num of Dest Networks displayed.
--------------------------------------------------------------------------------------------------
TYPE Legend:
I=Indirect Route, D=Direct Route, A=Alternative Route, B=Best Route, E=Ecmp Route,
U=Unresolved Route, N=Not in HW, F=Replaced by FTN, V=IPVPN Route, S=SPBM Route
PROTOCOL Legend:
v=Inter-VRF route redistributed
WM-01:1#
WM-01:1#ping 10.1.1.1
10.1.1.1 is alive
WM-01:1#
WM-01:1#show isis logical-interface

========================================================================================================================
ISIS Logical Interfaces
========================================================================================================================
IFIDX NAME ENCAP L2_INFO TUNNEL L3_TUNNEL_NEXT_HOP_INFO
TYPE PORT/MLT VIDS(PRIMARY) DEST-IP PORT/MLT VLAN VRF
------------------------------------------------------------------------------------------------------------------------
1 -- IP -- -- 10.1.1.1 Port1/1 10 GlobalRouter

------------------------------------------------------------------------------------------------------------------------
1 out of 1 Total Num of Logical ISIS interfaces
------------------------------------------------------------------------------------------------------------------------
WM-01:1#
WM-01:1#show isis interface

====================================================================================================
ISIS Interfaces
====================================================================================================
IFIDX TYPE LEVEL OP-STATE ADM-STATE ADJ UP-ADJ SPBM-L1-METRIC
----------------------------------------------------------------------------------------------------
10.1.1.1 pt-pt Level 1 UP UP 0 0 20000

--------------------------------------------------------------------------------
1 out of 1 Total Num of ISIS interfaces
--------------------------------------------------------------------------------

WM-01:1#
WM-01:1#show isis int-l1-cntl-pkts

====================================================================================================
ISIS L1 Control Packet counters
====================================================================================================
IFIDX DIRECTION HELLO LSP CSNP PSNP

----------------------------------------------------------------------------------------------------
10.1.1.1 Transmitted 421 0 0 0
10.1.1.1 Received 0 0 0 0

WM-01:1#

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Rajshekhar_Bira — Mon, 18 Jun 2018 10:53:00 GMT

Due to lack of Data plane support in current VOSS GNS3, FE adjacency will not work as it needs VxLAN encap.

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Ludovico_Steven — Mon, 18 Jun 2018 12:10:00 GMT

The CPU generates the ISIS Hellos for SPB (which is why you can get regular ISIS adjacencies working with the VOSS VM), but with FE (on VSP8k & 7200) it is the hardware which handles the VXLAN encap to make FE happen; there is currently no hardware emulation in the current VOSS VM, so it won't work.
Looking at your config, you seem to be setting an area with many zeros.. usually 49.0001 is what we recommend.I also notice they your ISIS source IP and your Tunnel source address are the same. Which means you are trying to run FE on a GRT loopback.Although possible, we don't normally recommend that. It is best to place the tunnel endpoint in a dedicated FE VRF (either a physical brouter IP or a loopback IP if you wish).
In a fabric design the GRT is usually always IP-Shortcuts enabled, as this is the recommended way to handle inband fabric management of L3 capable SPB nodes (i.e. VSPs).If you are running FE over a L3 cloud, you will need some IP routes in order to reach your tunnel destinations; and if your tunnel endpoint IPs are loopbacks you will always need IP routes, even if doing FE over a flat L2 ELAN cloud.Now, if these routes are in GRT, and at the same time you enable GRT IP Shortcuts to advertise its routes over ISIS (effectively using the same routing instance for FE underlay and SPB overlay) you have to be extremely careful that the routes that FE needs must never get advertised over ISIS. Otherwise your FE ISIS adjacencies will fall.

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Erol_Asim — Mon, 18 Jun 2018 16:18:00 GMT

Hi Ludo,

Thanks for the response.

I got 2x 8k's directly attached together and configured FE - adjacency was up - tested to make sure it works. Instead of loopbacks, I used brouter ports for the source/dest of the ip tunnel.

When I put 1x L2 switch in the middle, with only the brouter vlan spanning across, although the 2x 8k's can ping each other, the adjacency is down. I have added accept command to prevent the FE subnet from being learned by ISIS as I do have IP shortcuts enabled, but no adjacency. The brouter subnet does not exist anywhere else in the network.

I would expect if I can ping across considering there is no routing on the switches in-between and both sides are on the same subnet, the adjacency should come up? Am I missing something here?

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Ludovico_Steven — Mon, 18 Jun 2018 17:20:00 GMT

Are your 8ks VMs ? Or real units ?

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Erol_Asim — Mon, 18 Jun 2018 17:21:00 GMT

Real 8k's this time around

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Ludovico_Steven — Mon, 18 Jun 2018 17:42:00 GMT

There must be something wrong with your config. Maybe share them ?
Here are some snippets..
For the underlay:

ip vrf spboip vrfid 511interface GigabitEthernet 2/38 vrf spboip brouter port 2/38 vlan 900 subnet 172.16.101.81/24exitrouter vrf spboip ip route 172.16.100.0 255.255.0.0 172.16.101.1 weight 10exit

And for the overlay:

router isis ip-tunnel-source-address 172.16.101.81 vrf spboipexitlogical-intf isis 1 dest-ip 172.16.100.41 name to_VSP4000-1 isis isis spbm 1 isis spbm 1 l1-metric 2000 isis enableexit

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Erol_Asim — Mon, 18 Jun 2018 19:21:00 GMT

Still no luck. If I directly attach the 8k's - adjacency comes up. If I put a switch inbetween, nothing.

SSTN:

SSTN-VSP-CORE:1(config)#ping 192.168.0.130 vrf spbFE
192.168.0.130 is alive
SSTN-VSP-CORE:1(config)#ping 192.168.0.131 vrf spbFE
192.168.0.131 is alive
SSTN-VSP-CORE:1(config)#show run

spbm
spbm ethertype 0x8100
ip vrf spbfe vrfid 511

router isis
spbm 1
spbm 1 nick-name 1.01.01
spbm 1 b-vid 4051-4052 primary 4051
spbm 1 multicast enable
spbm 1 ip enable
spbm 1 smlt-virtual-bmac 82??84:00:01:ff
spbm 1 smlt-peer-system-id 82bb.8401.0201
exit

ip prefix-list "IST" 10.163.75.32/30 id 1 ge 30 le 30
ip prefix-list "FE" 10.163.139.128/26 id 3 ge 26 le 32

vlan create 3998 name "IST-CORE-CW" type port-mstprstp 1
vlan i-sid 3998 9013998
interface Vlan 3998
ip address 10.163.75.33 255.255.255.252 0
exit
vlan create 4051 name "SPB-BACKBONE1" type spbm-bvlan
vlan create 4052 name "SPB-BACKBONE2" type spbm-bvlan

virtual-ist peer-ip 10.163.75.34 vlan 3998

interface GigabitEthernet 1/2
no shutdown
vrf spbfe
brouter port 1/2 vlan 1002 subnet 192.168.0.130/255.255.255.192 mac-offset 108
no spanning-tree mstp force-port-state enable
exit

interface GigabitEthernet 1/17
default-vlan-id 0
name "IST"
no shutdown
isis
isis spbm 1
isis enable
no spanning-tree mstp force-port-state enable
no spanning-tree mstp msti 62 force-port-state enable
exit

route-map "suppressIST" 1
no permit
enable
match network "IST"
route-map "suppressIST" 2
permit
enable
route-map "suppressFE" 1
no permit
enable
match network "FE"
match protocol isis
exit

router vrf spbfe
ip route 192.168.0.128 255.255.255.224 192.168.0.131 weight 10
exit

interface loopback 1
ip address 1 10.163.75.1/255.255.255.255
ip ospf 1
exit

router isis
sys-name "***"
ip-source-address 10.163.75.1
ip-tunnel-source-address 192.168.0.130 vrf spbFE
is-type l1
system-id 82bb.8401.0101
manual-area 49.0001
exit
router isis enable

logical-intf isis 1 dest-ip 192.168.0.131 name "Tunnel-to-WMH"
isis
isis spbm 1
isis spbm 1 l1-metric 2000
isis enable
exit

cfm spbm mepid 51
cfm spbm enable

router isis
accept route-map "suppressFE"
exit

router isis
redistribute direct
redistribute direct route-map "suppressIST"
redistribute direct enable
exit

WMH:

ping 192.168.0.131 vrf spbFE

192.168.0.131 is alive
WMH-VSP-CORE-01:1(config)#ping 192.168.0.130 vrf spbFE
192.168.0.130 is alive

spbm
spbm ethertype 0x8100

router isis
spbm 1
spbm 1 nick-name 2.01.01
spbm 1 b-vid 4051-4052 primary 4051
spbm 1 multicast enable
spbm 1 ip enable
spbm 1 smlt-virtual-bmac 82??84:00:09:ff
spbm 1 smlt-peer-system-id 82bb.8402.0201
exit

ip prefix-list "IST" 10.163.139.32/30 id 1 ge 30 le 30
ip prefix-list "FE" 10.163.139.128/26 id 3 ge 26 le 32
vlan create 3997 name "IST-CORE-WM" type port-mstprstp 1
vlan i-sid 3997 9023997
interface Vlan 3997
ip address 10.163.139.33 255.255.255.252 0
exit
vlan create 4051 name "SPB-BACKBONE1" type spbm-bvlan
vlan create 4052 name "SPB-BACKBONE2" type spbm-bvlan

virtual-ist peer-ip 10.163.139.34 vlan 3997

interface GigabitEthernet 1/2
no shutdown
vrf spbfe
brouter port 1/2 vlan 1002 subnet 192.168.0.131/255.255.255.192 mac-offset 3
no spanning-tree mstp force-port-state enable
exit

route-map "suppressIST" 1
no permit
enable
match network "IST"
route-map "suppressIST" 2
permit
enable
route-map "suppressFE" 1
no permit
enable
match network "FE"
match protocol isis
exit

router vrf spbfe
ip route 192.168.0.128 255.255.255.224 192.168.0.131 weight 10
exit

router isis
sys-name "WMH-VSP-CORE-01"
ip-source-address 10.163.139.1
ip-tunnel-source-address 192.168.0.131 vrf spbFE
is-type l1
system-id 82bb.8402.0101
manual-area 49.0001
exit
router isis enable

logical-intf isis 1 dest-ip 192.168.0.130 name "Tunnel-to-SSTN"
isis
isis spbm 1
isis spbm 1 l1-metric 2000
isis enable
exit

cfm spbm mepid 91
cfm spbm enable

router isis
accept route-map "suppressFE"
exit

router isis
redistribute direct
redistribute direct route-map "suppressIST"
redistribute direct enable
exit

isis apply redistribute ospf
isis apply redistribute direct

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Rajshekhar_Bira — Mon, 18 Jun 2018 19:34:00 GMT

May be you are hitting MTU problem. Make sure the switch in-between supports more than 1600 bytes of MTU size.

RE: Fabric Extend - cannot get adjacency over IP Tunnel

Roger_Lapuh — Wed, 20 Jun 2018 15:13:00 GMT

good point, if MTU is not sufficient, we don't allow the IS-IS session to come up.

RE: Fabric Extend - cannot get adjacency over IP Tunnel

huhuan — Sat, 24 Aug 2019 05:46:00 GMT

If isp supports MTU greater than 1594, without ONA, how to configure vsp4k?