ExtremeSwitching (EXOS)

Hey,

Can you validate that the following is a valid setup using MLAG.

Sites A and B contain two MLAG peers each.

There are two independent links between sites A and B, and in order to connect both sites, both peers can use MLAG at each end. (Each group of switches thinks that it is talking to one switch at the other end.)

This would make the two links active, thus double the bandwidth compared to STP which would always block one ISL. But I'm not sure about STP in this scenario.

I couldn't really validate nor not validate such a setup from reading the manual. Is it valid?

Anything that comes to mind about this?

Thanks.

Hi jeronimo,

I've encountered this when visiting one of the customers.
From MLAG perspective, it will forward the traffic, but IMO you don't benefit MLAG here. It's like having failover scenario due to diagonal links (A1-B2, A2-B1) outage.
I'm not sure right now if the ISC blocking filters be disabled due to this.

With such four switches you can also think of different non-port-blocking mechanism, it could be EAPS or A and B stacks + LACP... Or perhaps you would like to play with MSTP and manipulating the bridge priorities for every MST instance so for each VLAN root bridge is different one, so all links are used. But that sounds like a lot to do. ;)
All depends on place in the network and your goals and constraints.

Kind regards,
Tomasz

Hi jeronimo,

the topology is valid. No link needs to be blocked with MLAG while STP, EAPS, or similar would need to block at least one (logical) link. You might want to consider adding cross-links (upper left to lower right, lower left to upper right) if possible to optimize redundancy and fail-over speed.

Please see the GTAC Knowledge site for additional info:

• Explaining Two-tier MLAG design
• Key Design Points of an EXOS Multi-LAG
• How to configure MLAG in Extreme switches

Thanks,
Erik

Ok, thanks. Now I am wondering indeed what would be the use of using MLAG here. As Tomasz claims, there's no real benefit here. What would you do? Use MLAG for those ISLs or simply treat them as single links and let STP handle them? Cross-links are not possible right now.

Hi Paul,

the setup should work without the cross connections, we are using this exact setup in the network of the small company I work for (we are using older EXOS switches, not -G2, but this should still work). We have a switch pair on the office floor and another one in the basement connected via two-tier-MLAG using just two links.

Thanks,
Erik

Interesting this question came up - I was about to post about a near identical setup.

The benefit I see of this "back to back" MLAG is that both sides just see a LAG, so you have a true active-active situation for traffic in either direction. The alternative is EAPS or STP; which would work, and you could probably arrange multiple VLANs to get maximum use of both connections. But it is still more work and config compared to the MLAG example.

When I did some testing of this (well actually ended up deploying it with 2x X460G2s on one side, and 2x customer X440s on the other side), I was seeing some destination macs in the same L2 network that just didn't work. It had all the hallmarks of a situation where a LAG without LACP was unidirectional so a packet was hashed and sent down a link that just didn't work.

I can't see why this shouldn't work, and so long as you have LACP enabled (we do all remember to always enable LACP, don't we folks) you have a control protocol to deal with connectivity problems where the link may appear to still be up and avoid the blackhole situation I saw.

The "Two Tier MLAG design" topologically isn't quite the same as what we're describing here - it has the cross connections in place. Are they required to make this legal (I can imagine, given that I had packets going nowhere, that this would fix what I saw) - or should this "just work" with the simple two cables between two sets of switches setup?

Paul.

IMO single STP domain makes you benefit less as the link is blocked. MSTP might cover this issue, but more to configure. If those cross-links are not possible just for some time, perhaps you might leave it as is until it becomes full-blown MLAG (then there'll be not much to reconfigure). 🙂

The benefits are all links active and no extra protocol running.

I found using MLAG on both sides between switches quite a thought experiment, since you usually somehow associate MLAG with the "server"/"switch"-side and a port-channel/LAG with the "client"/host-side. In this scenario, MLAG is both...

This two-tier-MLAG construct is used in quite a few EXOS based networks, because all links are active as opposed to blocking links with STP or EAPS.

Paul has a point. I was testing back-to-back MLAG topology concerning all kinds of link failures:
* direct: the simplest failure: the port goes physically down
* indirect: 1. there is something in-between that isn't passing data anymore, or 2. there is nothing in-between so something on the other side has died but the link is still up

Now the challenge is of course an indirect link failure. Like when you "pause" the link in GNS3, when the ports stay up but data isn't passing. Whether you implement the cross connections or not, I found that when I pause one of the links, some traffic starts getting blackholed. Probably because in back-to-back MLAG there is nothing like LACP which would watch the links (or loop-protect like in STP which stops using the link if there are no more BPDUs).

Did your "Back-to-Back" design work out? I just asked GTAC, before finding this post, about this same setup and they told me that it wouldn't work.

That's interesting. But first, we should find out what everyone is talking about. Also if "back-to-back" means the same thing as "two-tier" and what exactly that is.

My post was mainly about this: https://gtacknowledge.extremenetworks.com/articles/How_To/Sample-configuration-for-two-tier-MLAG/ with the subtle difference that I don't have a full mesh but only one ISL each. So we already have two setups. What exactly did you ask from them?

Information in general about this subject seems very sparse, not many people seem to have a full grasp of this technology. I have the feeling before raising false expectations they will just tell you it won't work. In my case I was told by a local engineer during redesign of our previous DC network that this wouldn't be a problem. The full mesh would be recommended, but it would work either way.

Oh and to answer your question: yes, it does indeed work. It even works with a 3rd party RSTP "ring" attached to two of the four switches

I have a large customer that is moving away from EAPS due to several reasons, including issues with maintaining configs (i.e. consistency between switches in the rings), the poor link utilization (one link blocked), licensing and stability reasons. They are moving to an MLAG setup where the core distribution has multiple MLAG pairs and the local distribution (multiple for each core dist pair) consists of switch pairs running as MLAG peers. We asked Extreme TAC about the feasibility of running MLAG to MLAG with only the direct links (not full mesh with the X in the middle) and after some investigation on their side, even involving a meeting with us and some high-ranking TAC experts, they came back and said it would work and (I think officially) be supported.

If you think of it, each MLAG peer pretends to be one single switch presenting a LAG to the other end of the LAG. IF this is the case from one side of an Extreme MLAG pair to a server with dual NICs or if it is between two MLAG pairs shouldn't make a difference. Other vendors support this so it would be strang if Extreme wouldn't. That said, in the core I would still go for full mesh.

Someone said you wouldn't gain anything from an MLAG-MLAG setup if it's not full mesh, but I certainly don't agree. We get 2 x the capacity compared to both EAPS and STP unless we go through a lot of hassle to create multiple domains, and that makes for a very coarse load balancing indeed. The most important gain is that MLAG seems more stable than EAPS and STP, even if there are still scenarios where MLAG can create problems.

B.t.w. if you use VRRP in an MLAG pair, did you know that the fabric-routing allows both peers to route?

https://gtacknowledge.extremenetworks.com/articles/How_To/An-example-of-VRRP-fabric-routing-configuration-to-achieve-active-active-forwarding-routing-on-all-VRRP-routers

Another tip for you is that if you have two fiber pairs between two sites, you can still have four links, and have your full mesh if you like. Just use BiDi SFPs! CWDM is another alternative that is not very expensive today.

Please, people, STP in this day and age???? I always say STP was great in the 80's when it was invented, but today there are almost always better alternatives.

/Fredrik