topic Re: Mlag data link in ExtremeSwitching (EXOS/Switch Engine)

Mlag data link

wsalkos — Tue, 26 Sep 2023 08:22:10 GMT

Hi, I have 2 stacks of X670 devices

I am not very satisfied with the operation of the stack, for example the fact that when updating the stack, the entire stack has to be restarted, which causes a break.

I started thinking about implementing mlag.

I would like to stack MLAG between two stacks.

I'm reading mlag documentation: https://extremeportal.force.com/ExtrArticleDetail?an=000079895

but I'm wondering how to configure data link between stacks in mlag.

Let's assume the scheme as below, that we have a server connected to port access vlan 100 in stack 1, and another server also in vlan 100 but in stack2.

sorry for this diagram, it's just a preview

Should I tag VLAN 100 in the ISC connection or create some data link between the stacks?

I understand the idea of mlag when I have lacp on the server and I plug one link into one mlag peer and the other into the other, but what if I simply have a situation like the one above.

Should I use the isc connection as, say, a "trunk" or create another LACP connection between the stacks, but what does mlag say then?

Re: Mlag data link

Chris_H — Tue, 26 Sep 2023 12:47:01 GMT

For data you should just use the same physical connection that's being used as the ISC (and this is preferably a link aggregation going over both stacks for redundancy purposes)..

So let's say for example you use VLAN 100 for data between the two servers, then VLAN 100 would be added to the ISC link as tagged VLAN, in addition to a dedicated "ISC-VLAN" over which the two MLAG-peers can communicate and checkpoint their data to ensure they are properly synched up.

In your example the access-ports to the servers should NOT be configured as MLAG ports, as these servers are just single-homed and only local to one of the two MLAG peer-switches, so you wouldn't have any advantage of the MLAG configuration (i.e. SRV1 would be unreachable if the left stack is being rebooted/upgraded/goes down). The advantage of MLAG is obviously only applicable to devices that are connected to both of the MLAG peers simultaneously.

Re: Mlag data link

FredrikB-NN2 — Tue, 26 Sep 2023 12:51:10 GMT

In general, I would advice against using stacks all together in EXOS. This is due to oddities like the one you describe, loops on stack reboot and other stuff. I would recommend you to create two MLAG pairs and run an MLAG-to-MLAG connection between the two pairs. Depending on your site layout and distances, you could have each MLAG pair span both sites or have one MLAG pair per site.

Also, create a NORMAL LAG (mot an MLAG) that is used for the ISC connection. This is way better than a single ISC link and an alternate ISC link/VLAN. If you can, route the two individual links in the ISC different paths, even if they are placed in the same rack. Also, donät use adjacent ports, but spread them out as much as you can. I guess you have 6 x QSFP+ ports in the 670 (are those G2 versions?). If so, use the first and last ports for the ISC if you can spare two in each switch. This is in order to eliminate hardware dependencies. In some (almost all) switch platforms, there are components (PHY chips, ASICS etc.) that are common for several ports. If you get hit by a blockage or malfunction in one such component (it happens, trust me!) and that component serves both ISC links, you're toast. If you have the ports spread out, you minimize the risk of that happening.

When it comes to connection the MLAG pair to each other, you do not need to use a full mesh MLAG, but you can use single links like this:

Sw1===ISC===Sw2 <--- This is MLAG pair 1

Sw3===ISC===Sw4 <--- This is MLAG pair 2

Create a sharing in switch 1 with one single port in it (yes, a LAG of one port). Do the same in switch 2. Then, create an MLAG in both, referencing the LAGs you just created. Do the same on sw3 and sw4. Now, you can connect sw1 to sw3 and sw2 to sw4.

In the docs you will see that they use two ports from sw1 and two ports from sw2 and cross-connect them in a full-mesh style to sw3 and sw4. This is not needed, and verified by engineering.

Re: Mlag data link

wsalkos — Wed, 27 Sep 2023 10:32:25 GMT

thank you very much for the good advice! yes i have X670G2

to illustrate, because I don't know if I understand correctly, would you do it as follows?

but what if, for example, SW 1 or SW 3 fails?

I understand that if I have fifty vlans on each switch, then LAG(ISC) adds these vlans as tagged?

I thought it worked like this: on ports, e.g. 47 and 48, I do a lag and add an ISC vlan there as keepalive, and on ports, e.g. 49 and 53, I do a LAG between switches for data transfer and vlan tagging?

Re: Mlag data link

Chris_H — Wed, 27 Sep 2023 12:49:59 GMT

I think what FredrikB-NN2 means is a two-tiered MLAG design as explained here: https://extremeportal.force.com/ExtrArticleDetail?an=000082635

As mentioned the cross-connection for full mesh (P3 in the drawing) is not necessarily needed, but still recommended per design.

Re: Mlag data link

FredrikB-NN2 — Fri, 29 Sep 2023 12:41:52 GMT

Yes, correct. In the drawing posted on Wednesday, you have all four with ISC between them, which is actually a valid config (a switch can have two MLAG peers), but far from recommended!!! Just stay away from EXOS/SE 32.x for the MLAG part and any sharing until those problems have been fixed (working with TAC from a customer perspective on that). Latest 31.7.2 would be your choice.