EXOS MLAG/LACP question / clarification

[Frank]

I'm trying to get a clearer understanding of when to use MLAG with LACP and when not. I did read the note in https://extremeportal.force.com/ExtrArticleDetail?an=000079895 that reads "if the downstream device (e.g. switch, server, etc..) is using LACP LAG sharing on its ports, it will be necessary to enable LACP LAG on the MLAG ports", but I'm still a little fuzzy - probably because I don't understand the "black magic" fully.

Here's my layout - we're running EXOS 16.x:
ISC (ports 1:24/2:24) BD8800-1 ==== BD8800-2 1:1| 2:1\ 1:1/ |2:1 | \ / | | X | | / \ | 23| 24/ 23\ |24 Switch-1 Switch-2 5| 5| | | | | Server (Linux or Microsoft) with LACP share

Shared port group between the BD8800s for the ISC (enable sharing 1:24 grouping 1:24,2:24)
1-port shared "port group" to Switch-1 on the 8800 (enable sharing 1:1 grouping 1:1)
Same to Switch 2 (enable sharing 2:1 grouping 2:1)
Shared port group on Switch-1/2 to the BD8800s (sharing 23 grouping 23,24)
Single-port share-group on Switch-1 and Switch-2 on the port 5 to the server

mlag: enable mlag port 1:1 peer "8800-1/2" id 123 (on the 8800s)

On top of that: The BD8800s are the "default gateway" in a VRRP configuration for the vlan that the attached server is in (in case it matters)

Question 1:
In the scenario above, what shares on which switches/shares would have to be done with LACP - for instance "enable sharing X grouping ... L3_L4 LACP" (since I'd like to use L3_L4 on the server side)?

Question 2:
If I were to add an ISC link between Switch-1 and Switch-2, would the only LACP share configuration be on Switch-1/2 (and not on BDs)?

Question 3:
Are there any advantages/disadvantages to using "LACP" as opposed to not using it and going with Extreme's "pure purple magic"?


Thank you for looking at this!

[Tomasz]

Hi Frank,

If your Switch1 and 2 had an ISC, you can create MLAG, so from the Server perspective they will be a single device (virtual MAC, somewhat similar to virtual IP with VRRP concept). Then there is a point for aggregating uplink from Server to Switch1 and Switch2. At the moment (as per your diagram) this LACP on server side shouldn't be fully sucessful as on both ends the Server will see two different LACP keys (while it needs one, what is achievable thru MLAG).
Your 2-tier MLAG between switches and BDs is quite nice, but it doesn't do anything with Switches/Server communication. If you had ISC between the switches, you'd take advantage and have a possibility to terminate LACP/LAG from your server to two separate switches and have it presented to the Server as single logical link, that's what MLAG is about. 😉
Having situation like right now, LACP won't work 100% and static LAG would work (I'm only concerned about having the same MAC address learnt in two separate spots), but you might also experience sub-optimal flow handling - ISC would also pass data VLANs, so whenever some packet comes from the Server due to hashing algorithm to the left switch, but something is attached to the right switch, it can travel across ISC directly to the right switch without doing some workarounds and increasing vertical links utilization.
Once your ISC breaks, some sort of disruption might be presented AFAIK (e.g. FDB Checkpointing that is about p2p exchange of learned source MAC addresses). That's why it is a good practice to have it built on aggregation as well. And, just as a sneak peek, in SPB-based Fabric networks there is a feature called 'Virtual IST' that allows to run MLAG (called SMLT) successfully as long as there's ANY connection between peers. Cool, huh? 🙂
Another example: once I talked to a customer who had 'officially approved' two-tier MLAG design with only ISCs and vertical links, without diagonal ones. I was struggling with myself to eventually see, that yes - it will work, but like having the MLAG in failed links condition, so less bandwidth and resiliency in fact.
If it's not an issue, just go for an ISC between Switch1 and Switch2.

Hope that helps,
Tomasz

[Frank]

David, I'm not sure I fully understand. You're correct in that (currently) Switch-1/2 do not have an ISC link between them. I thought the mlag on the BD8800 would/should take care of the proper packet paths.

Let's say I have a full two-tier MLAG configuration, meaning an ISC link between switch1/2, appropriate config changes on the BD8800s (need to share/group 1:1 and 1:2), and of course define an mlag port 23 peer switch1/2:
What would happen if the ISC link between switch-1/2 dies? Wouldn't I be in the same situation as I'm in now?

[David_Choi]

Hi Frank,

It looks like the switch-1/2 are not configured with MLAG currently. Is it correct? As they are not in MLAG, you can not configure LACP on server side. This is because the switch-1 and 2 are physically / logically different hardware.

[Frank]

Oh, so all I'd need to do is to configure on Switch-1 and 2 something like

code:

enable sharing 5 grouping 5 l2_l3 lacp

And then I'm done with it? That would be downright easy!

Thank you so much for the easy to understand explanations - I got stuck in a loop between "overthinking", "read all kinds of technical docs" and "head exploded".

[OscarK]

The only thing LACP does is negotiate members of a LAG.
LACP will make sure the right ports are in the LAG and patch mistakes are not causing weird problems as the ports will not join the LAG.

LACP does not do anything more, it does not determine any automatic load sharing alghorithm, that is configured manually with the configured sharing algorithm.

The only disadvantage of LACP is that the lacp packets are processed by the CPU. If the CPU gets really busy or congested LACP packets might get lost resulting in ports being removed from the LAG.