X590 - MLAG - best practice for ISC

[Peter11]

Hallo Community,

I wish you all a happy new year.

We have two X590s and want to configure them with MLAG. A QSFP28 port (here QSFP28_1) is planned for each X590 for the uplinks to the core switch.

We have several options for the ISC connection between the X590:

a) X590_1 - QSFP28_2 <== Direct Attach ==> X590_2 - QSFP28_2
      [high performance and no hardware redundancy]

b) X590_1 - 10G ports <== trunk with 2 x or 4 x ==> X590_2 - 10G ports
      [medium performance with hardware redundancy, reduction of usable 10G ports]

c) a mixture of a) and b) or completely different

Is there perhaps a recommendation for the throughput of the ISC connection?

What are your experiences and recommendations for network design?

Thank you very much and best regards
Peter

[Stefan_K_ · Accepted solution]

Hi Peter,

For an ISC it's always: Redundancy over performance, at least for me. ISC is only needed when the direct uplinks to the Core fails on one X590. 

X590 are a bit castrated unfortunately when it comes to your scenario. I always did it like this:
- 2 to 4 SFP+ for ISC and 1 (or 2) QSFP28 Uplinks to Core on each X590

A mixture (1x 100G + 1x10G as redundancy) isn't possible since you can't mix 10G and 100G ports in the same LAG. There was a change regarding this in EXOS 32.5: "Version 32.5 adds support for ports with different max speed capabilities to be part of the same Link Aggregation Group (LAG)."
But I think this is only related to the maximum possible speed and not the actual, current speed of a port... Haven't tested this yet and didn't find something regarding this in the User Guide.

Best regards
Stefan

[Peter11]

Hi Stefan,

Thanks a lot for your quick answer. It all sounds logical and confirms my gut feeling.

Can you possibly estimate how much traffic goes over the ISC connections if the uplinks are active on both X590?
This allows me to better calculate the number of ISC connections (2 or more).

Best regards
Peter

[Stefan_K_]

Hi Peter,

afaik it's only:

- Broadcast and Multicast, which then gets blocked by the other X590

- Control Traffic / Health Check between the Peers

The User Guide confirms this:

When at least one MLAG peer port is active, the upper layer software initiates a block of traffic that ingresses the ISC port and needs to be forwarded to the local MLAG ports. This is considered to be the steady state condition. In normal steady state operation most network traffic does not traverse the ISC. All unicast packets destined to MLAG ports are sent to the local MLAG port only. However, flood and multicast traffic will traverse the ISC but will be dropped from MLAG peer port transmission by the ISC blocking filter mechanism. The ISC blocking filter matches all Layer 2 traffic received on the ISC and blocks transmission to all MLAG ports that have MLAG peer ports in the active state.

---

EXOS User Guide: Configuring Slots and Ports on a Switch > MLAG > MLAG Overview > ISC Blocking Filters

2x 10G is totally fine if you can live with this bandwidth in case of an issue on the mlag ports.