Header Only - DO NOT REMOVE - Extreme Networks

Two tier MLAG


Userlevel 4
Create Date: Sep 22 2012 7:42AM

Can I possibly request urgent help with a two tier MLAG solution that I am trying to PoC. I basically have four X460s connected together in a full mesh. Two switches will emulate a Core and two to emulate edge switches.

1. Core1 connected to Core2 across an ISC consisting of two ports aggregated in a regular LAG.
2. Edge1 connected to Edge2 across an ISC consisting of two ports aggregated in a regular LAG
3. Core1 connected to Edge1 (2ports LAG) and Edge2(2ports LAG)
4. Core2 connected to Edge1 (2ports LAG) and Edge2(2ports LAG)

My question is how and where would I setup the LAG and MLAG between the Core and Edge. Here is my present config

1. On the Edge1 and 2 switches, the ports going to both Core switches are setup as a regular LAG.
2. On the Core1 and 2 switches, the ports going to the Edge1 switch are setup as a MLAG with id 1 and the ports going to Edge2 are setup with MLAG with id 2.
3. I was under the impression that the connection from each of the Core to both Edge switches could be setup as a MLAG on the Edge switches as well so that you would have MLAG at both ends.

There is presently a loop and the CPU percentage is around 40-50% which is not great. I cannot go to production with this.

I am following the concepts guide but it is sort of thin on details when it comes to the two tier config. I have two VLANS to emulate edge VLANS and they are propagated all across (on all uplinks - tagged). Any suggestions pls. (from Anush_Santhanam)

14 replies

Userlevel 4
Create Date: Sep 24 2012 1:15PM

Hello ExcaliburSorry for the late response but I was not on over the weekend.You are correct that you need to have MLAG set up on both sides of the connection. So MLAG on the cores going to the edge switches and on the edge switches back to the core. The links that are on the same switch, i.e. the edge sw 1 two ports to both core1 and core2 need to be a lag as normal. This needs to be done on any switch with more than one link. The opposite side in of that LAG that goes into two different switches need to have a MLAG.So you need a MLAG on the edge side from the core and on the core side from the edgeHope that helps.P (from Paul_Russo)
Userlevel 4
Create Date: Sep 26 2012 5:21AM

Great help with this. I managed to get this setup and working. One quick question please. In my PoC environment, when I run failover tests, one of the things that I am doing is to disable one ISC at a time, for example the ISC between Core1 and 2. At this point what I am observing is that the when the ISC is broken (port disabled), connectivity between the Core and Edge is disrupted and I can only ping one of the Edge switches. The cross links dont work. Is this expected. However when I power down a device, optimal resiliency is in place. Not sure why that is. I am running XOS 12.6.3 on all the boxes.

From a best practice perspective, would you recommend using either ELRP or ELSM with this or possibly EAPS. I think from memory you cannot use ELSM with MLAG. How about for the LAG (isc). In production, I am using VRRP which is something I could not replicate as I dont have the advanced edge license. Thank you again for the time and help. Really appreciated. (from Anush_Santhanam)
Userlevel 4
Create Date: Sep 26 2012 1:48PM

Hey ExcaliburBreaking the ISC is very bad and will result in bad behavior. This is why it is recommended that the ISC link be a LAG port so that a failure on one port does not result in two active connections in the core. Failing the ISC is not a valid test. Failing a link from the edge to the core or failing one of the cores is valid. The ISC can go away if have of the core is gone because there is no need to communicate with the other MLAG Peer.So if you are configuring MLAG on a chassis use LAG ports and spread them across modules. If you are using stacks use LAG and spread them across stack members. If you are using a single Summit switch then just use more than one port in a LAG preferable outside an 8 port group. For example port 1 and port 24.Does this help?P (from Paul_Russo)
Userlevel 4
Create Date: Sep 27 2012 4:34AM

Hello, Great help again. Thank you so much. That answers all my queries. In fact this is precisely what I am planning. I will be deploying Summitstack across floors for edge connectivity and the ISC will feature a LAG with multiple ports. For the Core, I am planning on aggregating across modules as well. This almost is a confirmation of the design I am putting in. I just wanted to ensure that I was covering all corners. Also, if I may trouble you one more time, I am guessing that I dont mess around with ELSM or ELRP and leave things as is. Great help and thank you for the time. (from Anush_Santhanam)
Userlevel 4
Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

Why is ELRP not helpful with MLAG ? Yesterday we did a "run failover" on a stack that leads to a misconfigured former master booting with default image. This former master was configured with two LAGed uplinks to a MLAG core pair. After booting with default image we got a loop that bring the complete network down ! The two MLAG cores lost connection to all edge switches and several servers. Would ELRP not be helpful in a situation like this ? Is there a way to disable only the affected MLAG peer ports so that the other ports are still operating ? If the complete VLAN ports are disabled by ELRP it might be possible to exclude always one peer port which will also prevent the loop.....
Userlevel 7
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

Hi,
LACP would be better in that case. With it enabled, the loop would not have been possible in your case. ELRP is great, but it's for preventing a loop on the access side.
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

Is this also true for the Cisco switches we connected to the Extreme MLAG cores with static LAG. At the moment they are all totally freezing if we have a loop on one MLAP peer and we have to unplug mains to repair this. If we use LACP for them too would that avoid it ? So ELRP would only be useful for the Extreme Edge switches we use or should be activate STP on them as with the Cisco ́s ?
Userlevel 7
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

LACP should be configured for every switches connected to the MLAG Core. That way, you are less likely to create a loop, or a blackhole, because of wrong cabling, misconfiguration or config loss. If LACP is not present on the other end of the link, it doesn't come up.

As for choosing ELRP or STP, it will depend on what scenario you want to be protected from. Just be sure to never block Uplinks on access switches. In the event you create a loop by connected one edge to another edge, prefer to block the direct edge to edge link rather than the uplink on one edge.
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

Ok, I see. If we configure the ISC connection with LACP is it still possible to create static MLAG peers together with the LACP peers ? We have some vSphere Host with standard vSwitches that can only use static LAG (Route based on IP Hash) and we want to connect them to the MLAG core switches too.
Userlevel 7
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

ISC is totally independent.
Indeed, you need vSphere 5.1 to have LACP. If you can't, you have to stick to static LAG with all the risks that it implies...
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

In this case we only have an Enterprise license without the Plus...That means no distributed switch. But a host cannot create a loop anyway, so it should be O.K. to go with LACP for the switches only.....
Userlevel 7
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

never say never 🙂
I have seen servers with 2 NIC starting to bridge between them, even though not configured for it.

But I agree. Configure LACP on NNI side and use STP or ELRP on the UNI side. That should be good enough to prevent 99% of loop potential.
EtherNation User wrote:

Create Date: Sep 27 2012 11:51AM

Hey ExcaliburYes you are correct you do not need either ELSM or ELRP in MLAG. ELSM is great for protecting against a neighbor switches CPU getting over run and making a link go down and is great for EAPS/ERPS. But in this case doing that will not help with MLAG.Hope that helps.P (from Paul_Russo)

Played a little with ELRP today and it looks really nice. So why should one use it only on the access side ? If we run it on the MLAG core pair and exclude the ISC it should also prevent any loops....

ELRP is so easy to setup that I asked myself why I should worry about LACP and STP (at least on the Extreme switches). Is there any disadvantage using it ?

Reply