S Series VXLAN LAB setup

[Andre_Brits_Kan]

Hi

I am busy playing with VXLAN between two S series switches in my LAB and have some problems.

The two S series switches connect to each other across a L3 routed network.
I have configured the VXLAN tunnel and that seems to be working fine.

VXLAN Setup for "SSA One":

interface tun.0.1 tunnel mode vxlan l2 tbp.0.1
tunnel source 172.16.1.1
no shutdown
exit

set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.2

set vlan egress 100 tbp.0.1 tagged
set vlan egress 100 ge.2.48 untagged (User Port)

VXLAN Setup for "SSA Two"

interface tun.0.1 tunnel mode vxlan l2 tbp.0.1
tunnel source 172.16.1.2
no shutdown
exit

set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.1

set vlan egress 100 tbp.0.1 tagged
set vlan egress 100 tg.1.24 untagged (User Port)

With the above configuration, devices connected to both S series switches in vlan 100 can ping each other.

-----------------------------------------------------------------------------------------

For the next part of my lab I would like to test routing from the remote site (SSA Two) to a network that is connected to "SSA one"

For this I added another vlan on "SSA one" and created a routed interface.

For users in vlan 100 (the VXLAN vlan) I have specified 192.168.100.1 (Vlan 100 interface on "SSA One") as the default gateway.
Clients connected the "SSA one" in vlan 100 can ping any device in vlan 10 and any device in vlan 10 can ping any device connected to "SSA one" in vlan 100.

The problem I have is with the remote clients.
The Remote clients connected to "SSA Two" in vlan 100 can ping any other device in vlan 100 (Both SSA one and SSA Two). These clients can also ping the Vlan 10 interface of "SSA One" (192.168.10.211), but they can not ping any other device in vlan 10.
The Vlan 10 devices can not ping any of the remote vlan 100 (SSA two) clients.

Why is this?

If I extend vlan 100 from "SSA One" to an external router that routes traffic for both Vlan 100 and Vlan 10 then every client can reach every other client.
Only when I use one of the terminating VXLAN SSA's for routing can the remote client not route out.

Hope this makes sense....

Regards

[Andre_Brits_Kan]

The reason for my LAB is that we have a client with two Data Centers.
I would like to extended some of the server vlans across the two datacentres.
The reason for testing the routing to a send outside vlan is that the two DC's connect to a very big OSPF cloud, so these VXLAN vlans must be able to route to the clients in the OSPF cloud....

[Andre_Brits_Kan]

Hi Erik

Thank you for your efforts here.
I found exactly the same results.

Only once I add vlan 10 to the Tunnel config can every device see every other device.

SSA One
set tunnel logical-switch create name Switch2
set tunnel map logical-switch Switch2 keyword 110 vlan 10
set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch2 ip-address 172.16.1.2
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.2

SSA two
set tunnel logical-switch create name Switch2
set tunnel map logical-switch Switch2 keyword 110 vlan 10
set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch2 ip-address 172.16.1.1
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.1

[Erik_Auerswald]

I achieve full connectivity by adding VLAN 10 to the VXLAN overlay:

SSA One
set tunnel logical-switch create name VXLAN_LAB set tunnel map logical-switch VXLAN_LAB keyword 100 vlan 100 set tunnel logical-switch create name VXLAN_10 set tunnel map logical-switch VXLAN_10 keyword 10 vlan 10 set tunnel remote-vtep logical-switch VXLAN_LAB ip-address 172.16.2.2 set tunnel remote-vtep logical-switch VXLAN_10 ip-address 172.16.2.2
SSA Two
set tunnel logical-switch create name VXLAN_LAB set tunnel map logical-switch VXLAN_LAB keyword 100 vlan 100 set tunnel logical-switch create name VXLAN_10 set tunnel map logical-switch VXLAN_10 keyword 10 vlan 10 set tunnel remote-vtep logical-switch VXLAN_LAB ip-address 172.16.1.1 set tunnel remote-vtep logical-switch VXLAN_10 ip-address 172.16.1.1That is unexpected. I would call this a bug. 😞

[Erik_Auerswald]

To recap my tests: with VXLAN, layer 2 connectivity is working. All three test PCs can reach all other PCs and the SVIs in the same VLAN, including the VXLAN extension for the VLAN. But there is no connectivity between User3 and User2. Using GRE L2 tunneling instead of VXLAN L2 tunneling works.

That differs from the initial description by Andre, where User1 did not have connectivity to User3.

[Erik_Auerswald]

Hm, I have changed the working GRE L2 setup to use VXLAN instead, and it still works fine stopped working . Id o not see the problem in your config.

The PC connected to VLAN 100 on SSA One can ping both the VLAN 100 PC on SSA Two (L2 over VXLAN tunnel between SSA One and Two) and the VLAN 10 PC on SSA One (L3 forwarding on SSA One).

I have enabled jumbo frames to support a 1500 byte MTU over the VXLAN tunnel, but this is not needed for basic ping connectivity.

The relevant configuration is as follows (I have changed some IP addresses and use different ports):

SSA One
configure terminal ! interface loop.0.1 ip address 172.16.1.1 255.255.255.255 primary no shutdown exit interface vlan.0.10 ip address 192.168.10.1 255.255.255.0 primary no shutdown exit interface vlan.0.100 ip address 192.168.100.1 255.255.255.0 primary no shutdown exit interface vlan.0.999 ip address 10.1.1.0 255.255.255.254 primary no shutdown exit interface tun.0.1 tunnel mode vxlan l2 tbp.0.1 tunnel source 172.16.1.1 no shutdown exit ! # Static routes configured on routed interfaces ip route 172.16.2.2/32 10.1.1.1 interface vlan.0.999 1 ! exit # port set port disable lag.0.1-62 set port disable tbp.0.2-62 set port disable ge.1.1-4,8-48 set port disable tg.1.1-4 set port jumbo enable ge.1.6 set port vlan ge.1.5 100 set port vlan ge.1.6 999 set port vlan ge.1.7 10 # spantree set spantree portadmin tbp.0.1 disable # tunnel set tunnel logical-switch create name VXLAN_LAB set tunnel map logical-switch VXLAN_LAB keyword 1000 vlan 100 set tunnel remote-vtep logical-switch VXLAN_LAB ip-address 172.16.2.2 # vlan set vlan create 10,100,999 set vlan name 10 LOCAL_CLIENTS set vlan name 100 DISTRIBUTED_CLIENTS set vlan name 999 WAN clear vlan egress 1 tbp.0.1;ge.1.5-7 set vlan egress 10 ge.1.7 untagged set vlan egress 100 tbp.0.1 tagged set vlan egress 100 ge.1.5 untagged set vlan egress 999 ge.1.6 untagged
SSA Two! interface loop.0.1 ip address 172.16.2.2 255.255.255.255 primary no shutdown exit interface vlan.0.999 ip address 10.1.1.1 255.255.255.254 primary no shutdown exit interface tun.0.1 tunnel mode vxlan l2 tbp.0.1 tunnel source 172.16.2.2 no shutdown exit ! # Static routes configured on routed interfaces ip route 172.16.1.1/32 10.1.1.0 interface vlan.0.999 1 ! exit # port set port disable lag.0.1-62 set port disable tbp.0.2-62 set port disable ge.1.1-4,7-48 set port disable tg.1.1-4 set port jumbo enable ge.1.6 set port vlan ge.1.5 100 set port vlan ge.1.6 999 # spantree set spantree portadmin tbp.0.1 disable # tunnel set tunnel logical-switch create name VXLAN_LAB set tunnel map logical-switch VXLAN_LAB keyword 1000 vlan 100 set tunnel remote-vtep logical-switch VXLAN_LAB ip-address 172.16.1.1 # vlan set vlan create 100,999 set vlan name 100 DISTRIBUTED_CLIENTS set vlan name 999 WAN clear vlan egress 1 tbp.0.1;ge.1.5-6 set vlan egress 100 tbp.0.1 tagged set vlan egress 100 ge.1.5 untagged set vlan egress 999 ge.1.6 untagged