ExtremeSwitching (EXOS)

Ok guys, I have a new site with two X450G2's running system version 22.5.1.7 and Advanced Edge License. I currently have a high speed WAN from our local ISP which is a private link back to our HQ. We obtained a cellular LTE device from another provider that assigned us a /30.

For proof of concept I am doing my testing on a spare X450G2 running the same 22.5.1.7 code but just the regular edge licence. I have the tunnel created and I can ping the endpoints which are the local and remote LTE router. But the provider of these LTE routers require us to create a GRE tunnel over them.

I created the tunnel on the HQ side which terminates into a Cisco 2901 router with this command:
interface Tunnel5
description LTE TUNNEL TO R BRANCH
ip address 192.168.205.33 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 10.200.0.1
tunnel destination 10.200.0.22
end

The LTE router at this HQ site is 10.200.0.1, and the LTE router at the "R Branch" we will call it, is 10.200.0.22. This Cisco Router can ping 10.200.0.22 without issue. The Extreme test switch can ping 10.200.0.1 without issue, so I know there is LTE connectivity.

Now for creating the GRE tunnels, this Cisco commands above work fine for our other branches that terminate into other Cisco 2901 routers. However this new site does not have any Cisco routers since we are doing routing in the switch.

Here is what I've attempted on the EXOS side for the tunnel:

Tunnel is plugged into port 2
sh port 2 vid
Untagged
Port /Tagged VID(s)
-------- -------- ------------------------------------------------------------
2 Untagged 222

sh vlan
Untagged ports auto-move: Inform
-----------------------------------------------------------------------------------------------
Name VID Protocol Addr Flags Proto Ports Virtual
Active router
/Total
-----------------------------------------------------------------------------------------------
Default 1 10.1.0.111 /16 -f----------T--------------- ANY 1 /26 VR-Default
Mgmt 4095 ------------------------------------------------ ANY 0 /1 VR-Mgmt
LTE 222 10.200.0.21 /30 -f-------------------------- ANY 1 /1 VR-Default

Other vlans excluded from this output...

create tunnel backuptunnel gre destination 10.200.0.22 source 10.200.0.21
configure tunnel backuptunnel ipaddress 192.168.205.34 255.255.255.252
enable ipforwarding tunnel backuptunnel
enable ipforwarding vlan LTE

sh iproute
Ori Destination Gateway Mtr Flags VLAN Duration
#s Default Route 10.1.0.254 1 UG---S-um--f- Default 100d:22h:32m:2s
#d 10.1.0.0/16 10.1.0.111 1 U------um--f- Default 100d:22h:32m:4s
#s 10.200.0.0/24 10.200.0.22 1 UG---S-um--f- LTE 0d:0h:19m:28s
#d 10.200.0.20/30 10.200.0.21 1 U------um--f- LTE 0d:1h:2m:17s
#s 192.168.205.0/24 192.168.205.33 1 UG---S-um--f- backuptunnel 0d:0h:25m:23s
#d 192.168.205.32/30 192.168.205.34 1 U------um--f- backuptunnel 0d:0h:40m:38s

I am expecting that this EXOS switch, 192.168.205.34 should be able to ping the Cisco side of the tunnel at 192.168.205.33, and the Cisco side should be able to ping the EXOS side as well. However the ping is failing.

Obviously the endpoints will be 192.168.205.33 at the branch and 192.168.205.34 at the HQ and we will push traffic over this if the primary WAN goes down (likely using OSPF).

Am I missing something, or did I configure something wrong?

Again this is just a test switch so we can do antying on it and not disrupt connectivity. But once I get it working I want to translate this config to the actual branch and drive the LTE device over there and plug it into port 2:49 (with gigabit SFP port). Port 1:49 is the current WAN handoff in its own transport vlan. The site has no connectivity issues today, but we want to throw in this LTE backup device just in case there is a fiber cut or other issue with our upstream provider. We have this configuration today with other sites, but all of those existing sites use either a Cisco 2901 or 2911 router. We are trying to eliminate that extra piece of equipment if at all possible.

Oh also second question, can I make a second tunnel off the same interface? In a perfect world I'd love to create a gre tunnel to our LTE device at our DR site, however I get an error message"
create tunnel drtunnel gre destination 10.200.0.22 source 10.200.0.21
Error: Failed to create tunnel 'drtunnel'


The Cisco 2901 at HQ has 5 tunnels on it from the same source, so I'm not sure why that can handle it but EXOS is throwing an error.

Nevermind. The problem was on the Cisco side. The Tunnel destination is supposed to be 10.200.0.21 NOT 10.200.0.22! DOH!

And I was able to create a second tunnel as well. Also the destination is the IP on the cisco routers at the other end. Not the device the vendor shipped us! I should of known. The GRE tunnel is established between the switch and the router. So the tunnel spans the endpoints... the little LTE devices the vendor shipped us are like a layer 2 bridge.

Nevermind. The problem was on the Cisco side. The Tunnel destination is supposed to be 10.200.0.21 NOT 10.200.0.22! DOH!

And I was able to create a second tunnel as well. Also the destination is the IP on the cisco routers at the other end. Not the device the vendor shipped us! I should of known. The GRE tunnel is established between the switch and the router. So the tunnel spans the endpoints... the little LTE devices the vendor shipped us are like a layer 2 bridge.

So to summarize the two tunnels on EXOS

create tunnel backuptunnel gre destination 10.200.0.1 source 10.200.0.21
configure tunnel backuptunnel ipaddress 192.168.205.34 255.255.255.252

create tunnel drtunnel gre destination 10.200.0.5 source 10.200.0.21
configure tunnel drtunnel ipaddress 192.168.205.38 255.255.255.252

Where 10.200.0.1 is the IP address on the Cisco Router ethernet interface that the LTE device (10.200.0.2) plugs into at HQ.
And 10.200.0.5 is the IP address on the Cisco Router ethernet interface that the DR site LTE device (10.200.0.6) plugs into.

The source is the switch vlan LTE, 10.200.0.21, which is placed untagged in one port, and that has a cable going to the vendors newly supplied LTE device that they assigned 10.200.0.22/30.

Cisco side HQ site
interface Tunnel5
description LTE TUNNEL TO R BRANCH
ip address 192.168.205.33 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 10.200.0.1
tunnel destination 10.200.0.21
end

Cisco side DR site
interface Tunnel6
description LTE TUNNEL TO R BRANCH
ip address 192.168.205.37 255.255.255.252
ip mtu 1400
ip tcp adjust-mss 1360
tunnel source 10.200.0.5
tunnel destination 10.200.0.21
end

Now we have IP connectivity between these endpoints, we can utilize them in our routing tables. We will brainstorm a more advanced routing protocol like ospf rather than ping protection on exos (paired with IP SLA on Cisco side).

Hi,

you should probably test large packet sizes and path MTU discovery over the GRE tunnel. Last time I checked EXOS did not support setting the MTU of a GRE tunnel (see GTAC Knowledge article Configure MTU size for GRE tunnel in EXOS).

It might be that too large packets are dropped when sent from the EXOS side. IP packets with DF bit set with size between 1401B and 1476B(*) should trigger an ICMP packet too big from the Cisco router, but bigger packets might be silently dropped by EXOS without jumbo frames enabled, or forwarded by EXOS if jumbo frames are enabled, but silently dropped on the way to the Cisco router.

It might be possible to work around that problem by reducing the MTU of all VLAN IP interfaces by the GRE encapsulation overhead of 24B(*) (see GTAC Knowledge article How to change MTU size on a VLAN?).

(*) assuming 20B IP header and 4B GRE header

Thanks,
Erik