S Series VXLAN LAB setup

[Andre_Brits_Kan]

Hi

I am busy playing with VXLAN between two S series switches in my LAB and have some problems.

The two S series switches connect to each other across a L3 routed network.
I have configured the VXLAN tunnel and that seems to be working fine.

VXLAN Setup for "SSA One":

interface tun.0.1 tunnel mode vxlan l2 tbp.0.1
tunnel source 172.16.1.1
no shutdown
exit

set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.2

set vlan egress 100 tbp.0.1 tagged
set vlan egress 100 ge.2.48 untagged (User Port)

VXLAN Setup for "SSA Two"

interface tun.0.1 tunnel mode vxlan l2 tbp.0.1
tunnel source 172.16.1.2
no shutdown
exit

set tunnel logical-switch create name Switch1
set tunnel map logical-switch Switch1 keyword 111 vlan 100
set tunnel remote-vtep logical-switch Switch1 ip-address 172.16.1.1

set vlan egress 100 tbp.0.1 tagged
set vlan egress 100 tg.1.24 untagged (User Port)

With the above configuration, devices connected to both S series switches in vlan 100 can ping each other.

-----------------------------------------------------------------------------------------

For the next part of my lab I would like to test routing from the remote site (SSA Two) to a network that is connected to "SSA one"

For this I added another vlan on "SSA one" and created a routed interface.

For users in vlan 100 (the VXLAN vlan) I have specified 192.168.100.1 (Vlan 100 interface on "SSA One") as the default gateway.
Clients connected the "SSA one" in vlan 100 can ping any device in vlan 10 and any device in vlan 10 can ping any device connected to "SSA one" in vlan 100.

The problem I have is with the remote clients.
The Remote clients connected to "SSA Two" in vlan 100 can ping any other device in vlan 100 (Both SSA one and SSA Two). These clients can also ping the Vlan 10 interface of "SSA One" (192.168.10.211), but they can not ping any other device in vlan 10.
The Vlan 10 devices can not ping any of the remote vlan 100 (SSA two) clients.

Why is this?

If I extend vlan 100 from "SSA One" to an external router that routes traffic for both Vlan 100 and Vlan 10 then every client can reach every other client.
Only when I use one of the terminating VXLAN SSA's for routing can the remote client not route out.

Hope this makes sense....

Regards

[Erik_Auerswald]

One difference is that I do not have an SVI for VLAN 100 on SSA Two...

Edit: Just tested this, it does not matter.

[Erik_Auerswald]

I'll try this out as well. So far I have replicated the GRE L2 tunnel setup, which works as intended, as you wrote above.

[Andre_Brits_Kan]

Ok so I have tested the same setup but this time useing just GRE tunnels

SSA one:
interface tun.0.1 tunnel destination 172.16.1.2
tunnel mode gre l2 tbp.0.1
tunnel source 172.16.1.1
no shutdown
exit
set spantree portadmin tbp.0.1 disable
set spantree portadmin tg.1.1 disable
set vlan egress 100 tbp.0.1 tagged

SSA Two:
interface tun.0.1 tunnel destination 172.16.1.1
tunnel mode gre l2 tbp.0.1
tunnel source 172.16.1.2
no shutdown
exit
set spantree portadmin tbp.0.1 disable
set spantree portadmin tg.1.1 disable
set vlan egress 100 tbp.0.1 tagged

All three users every other user and Router Interface.

So the question is why does this not work with VXLAN.

I am new to VXLAN but perhaps this operates in the same way as TRILL where you can not use the same switch to terminate trill and route....

[Andre_Brits_Kan]

Ok so I have simplified the diagram:

Summary:

User1
Vlan 100
Default Gateway = 192.168.100.1

User2
Vlan 100
Default Gateway = 192.168.100.1

User3
Vlan 10
Default Gateway = 192.168.10.211

Ping results:
User1
Ping to Default Gateway (192.168.100.1) = Successful
Ping to Vlan 10 Router Interface (192.168.10.211) = Successful
Ping to User 2 = Successful
Ping to User 3 = Successful

User2
Ping to Default Gateway (192.168.100.1) = Successful
Ping to Vlan 10 Router Interface (192.168.10.211) = Successful
Ping to User 1 = Successful
Ping to User 3 = Fail

User3
Ping to Default Gateway (192.168.10.211) = Successful
Ping to Vlan 100 Router Interface (192.168.100.1) = Successful
Ping to User 1 = Successful
Ping to User 2 = Fail


As previously mentioned when I move the routing function from the SSA to a external router (routing between vlan 10 and 100) all devices can ping all other devices.

I would have presumed that if User1 can access vlan 10 devices then and other device in vlan 100 with the same gateway would also be able to get to vlan 10.

I would have presumed that the SSA acts like a VXLAN L3 gateway when you terminate a tunnel on it.

This is my firt time in the ring the VXLAN so please correct me if I am wrong.
I will test the same setup with a GRE tunnel and report back.

Thx
Andre

[Erik_Auerswald]

Hi Andre,

what you are describing seems related to the usual case with VXLAN Layer 2 gateway functionality, where you need an external router to route between VXLANs.

Your setup is not the usual VXLAN data center use case, but more of a traditional Layer 2 point-to-point tunnel setup.

I would expect connectivity between 192.168.100.111 and 192.168.10.10 if both VLANs are routed on SSA One and both PCs use SSA One as default gateway. It would be interesting to know if adding the VXLAN tunnel breaks this.

The case for connectivity between 192.168.100.112 and 192.168.10.10 is unusual for VXLAN setups, because SSA One would need to decapsulate the VXLAN packet and then run the resulting frame through VLAN/SVI routing. The usual method is to route to the target VXLAN on the encapsulating switch, if you have a Layer 3 VXLAN gateway.

Does your setup work without an external router if you use a GRE L2 tunnel between the SSAs?

This is an interesting problem. 🙂

Best regards,
Erik