topic RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING in ExtremeSwitching (EXOS/Switch Engine)

ISLs in 2-tier MLAG setup with RSTP goto LISTENING

jeronimo — Sun, 25 Nov 2018 05:05:00 GMT

Hey,

Let's look at the following topology.

In order to accomodate a MAN loop I thought I'd set up a simulation using EXOS 22.5 featuring RSTP support in MLAG scenarios.

What is not clear is how to configure RSTP exactly, thus I have configured it very simply like this:

* DC_CORE_A1.60 # show conf stp
#
# Module stp configuration.
#
configure stpd s0 mode dot1w
configure stpd s0 priority 4096
enable stpd s0 auto-bind vlan vl-NetMgmt

Not even considering the MAN loop (links are disabled), the links between tiers A1-B1 and A2-B2 on A1 and A2 are no longer forwarding.

* DC_CORE_A1.60 # sh s0 port
Port Mode State Cost Flags Priority Port ID Designated Bridge
7 802.1D FORWARDING 200000 eDappw---- 128 8007 10:00:0c??8c:ce:1f:00
10 802.1D LISTENING 200000 eDappw---- 128 800a 10:00:0c??8c:ce:1f:00
11* 802.1D FORWARDING 180000 eDap-w---- 128 800b 10:00:0c??8c:ce:1f:00

* DC_CORE_A2.7 # sh s0 port
Port Mode State Cost Flags Priority Port ID Designated Bridge
10 802.1D LISTENING 200000 eDappw---- 128 800a 80:00:0c??8c:d0:e7:00
11* 802.1D FORWARDING 180000 eRapaw---- 128 800b 10:00:0c??8c:ce:1f:00
MLAG is set up correctly.
Disabling STP on either A1 or B1 solves the issue. But we will need STP to protect from the loop.
Would anyone have an idea what the problem is?

Thx

Bonus question: Why does it show "LISTENING" anyway? cause that's not a 802.1w state (DISCARDING/LEARNING/FORWARDING)

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

jeronimo — Sun, 25 Nov 2018 15:47:00 GMT

Update: Port 10 on B2 was disabled. This put ports 10 on both A1 and A2 to listening. That somewhat defies the active-active nature of MLAG.
If I disable any other port 10 on A1,A2 or B1 nothing similar happens.
When the problem appears, port 10 on A2 is flapping between LEARNING and LISTENING and the log repeately says " Fail to flush vlan inst 000f424a Switch port 10 error 1"

(Ports 7 on both devices toward the MAN loop are disabled. The picture doesn't show it but they are. For that matter, ignore the MAN loop.)

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

Zdeněk_Pala — Sun, 25 Nov 2018 15:47:00 GMT

I would go with EAPS to handle ring topology at the MAN and I would utilize ELRP to detect loop at the access. Any kind of STP based protocol is wrong at the ring topologies.

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

Erik_Auerswald — Sun, 25 Nov 2018 15:47:00 GMT

STP supports any topology. 😉

If a provider MAN ring is used it may not be possible to use EAPS over it. EAPS over third party switches has its own problems, including by default not opening the ring port after detecting a failure. CFM might help as well.

Anyway, the above scenario does not seem to require anything but MLAG with LACP.

But EXOS should understand that the MLAG-ISL is not part of the STP topology in an MLAG setup, and provide useful show output to help in verification and troubleshooting in addition to correct operation (the show output does not look as if the feature is working as it should, and I have not tested STP over MLAG ports with EXOS yet, so I do not know if it works or not).

A possible problem in the scenario above might be the EXOS RSTP requirement of an untagged VLAN on an RSTP enable port, but the ISL is supposed to have tagged VLANs only. (I do not use RSTP on EXOS, but only MSTP, because of this limitation.)

Thanks,
Erik

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

jeronimo — Sun, 25 Nov 2018 15:47:00 GMT

Well, the actual topology is more like the following and the switches forming the MAN loop are indeed 3rd party.

BTW the remark about EXOS RSTP carrier VLAN is another interesting and unanswered question. Lots of mysteries here 🙂

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

Zdeněk_Pala — Sun, 25 Nov 2018 15:49:00 GMT

With MLAG you do not need Loop protection between access and the core...

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

jeronimo — Sun, 25 Nov 2018 16:39:00 GMT

What is the newly introduced RSTP and MSTP support concerning MLAG in 22.5 and 22.6 good for?

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

Stephane_Grosj1 — Sun, 25 Nov 2018 17:34:00 GMT

Just like Zdenek, I don't see the need to run STP above MLAG. STP should only be used to avoid loops at the access (or ELRP instead of STP).

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

Erik_Auerswald — Mon, 26 Nov 2018 19:58:00 GMT

In general, STP over MLAG ports is required to e.g. detect loops between two access switches that are connected to the distribution (or core) via MLAG. That is, the loop is from one access port of one switch to another access port of the other switch (similar use case as for ELRP disabling the egress port).

RE: ISLs in 2-tier MLAG setup with RSTP goto LISTENING

jeronimo — Sat, 08 Dec 2018 06:28:00 GMT

The topology got lost 😞 I'm attaching it again.

Two things:

1) I believe the problem was due to intricacies with GNS3. When you disable the port on one switch (here B2 at the bottom right), unlike in the real world, the port on the other side of the link (A2) does not go down. So in this case the other side was still physically up but not receiving any BPDUs. Since ports 10 on A1,A2 and B1,B2 respectively form an MLAG, it somehow makes sense that both ports 10 on A1 and A2 went to LISTENING.

To simulate a physical link down you have to manually disable the ports on both ends of the link, in this case B2 and A2. In that case everything continues to work.

2) Concerning Erik's statement: "STP over MLAG ports is required to e.g. detect loops between two access switches"

Wouldn't it be enough to run STP locally on each of the concerned switches and put all ports (except uplinks) into edge-safeguard mode with bpdu-protection. In case of loops (no matter if local to the switch or from another access switch) they would be disabled. You would not actually need STP to traverse the MLAGs.