25G SFP28 in 5420 EXOS Stack no link in Backup/Standby Slots

[RobertD1]

Hello,

Failing to understand why 25G SFP28 will only work in the Master slot (ie port 1:52) in a stack of 5420s with EXOS. I want to distribute my uplinks to the core across two slots but cannot get the link up when the SFP28 in inserted in 2:52 or 3:52 in a stack. I have two 7432CQ with 100G breakout channelized for 4 x 25G. Have also tried with other Extreme 5420s and always the same issue with the Backup/Standby slots.

Would be great if someone could replicate and confirm the same?

Have opened a case already and going through the collection of information to help progress the case but was curious if anyone else has seen the same as seems quite a common topology, but maybe not so common to use 2 x 25G to the edge. 1G and 10G works fine as does the 25G DAC.

Feels like a bug and easy to reproduce, but why the choice of slot would matter I don't understand.

Thanks,

Rob

[RobertD1 · Accepted solution]

Disabling FEC so it matched both ends made the link come up immediately. 

[RobertD1]

I noticed that 1:52 has FEC enabled with clause 91 and 2:52 it is disabled. So, this is a candidate for the link failing in 2:52 I would say.

[FredrikB-NN]

You might want to check the FEC settings as this is a 25 G link.

show port 1:52,2:52,3:52 forward-error-correction no-refresh

Depending on the switch you connect to and the default in EXOS/VOSS you may need to change this setting. The defaults are, as always with extreme, anyone's guess. I wouldn't be surprised if the default is "off" for 1:x ports but something else for ports in additional members.

FEC is often recommended for SR optics and also for longer LR links. You need to have the exact same setting on both sides. The nanoseconds it adds to the latency are often not very deterring. The extra noise tolerance is between 2-6 dB, so can be quite useful. LLDP may or may not work with different settings, which may throw you off a bit.

/Fredrik

[RobertD1]

Thank you Fredrik, I will investigate this when I return to site in a few days time.