topic Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2 in ExtremeSwitching (EXOS/Switch Engine)

Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Mon, 29 Jan 2024 06:37:14 GMT

Hello,

we had some problems with the upgrade to 32.6.1.5-patch1-2 over the weekend. I wanted to ask if this is the case for you too.

When we updated the X690 the 40G ports no longer came up. They were active but we see no connection to the other switch. All 1G ports are working, only the 40G doesen't work. We use the new Image x690-32.6.1.5-patch1-2.xos (not ONIE). After we roll back .. everything is working.

When we updated the X440-G2 for some reason, the ssh service was no longer active after the upgrade restart. As soon as it was reactivated via the console port it worked.

Does anyone know the problem?

Best regards,

Markus

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

Stefan_K_ — Mon, 29 Jan 2024 07:33:28 GMT

That's why I stay with 31.7 until there is a new stable LTS Release in the 32.x branch.

Did the ports come up on both sides of the line or only on the X690? Was the port "blocked on the vlan"? Did you see lldp neighbors or mac addresses on the port?

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Mon, 29 Jan 2024 10:14:06 GMT

Everything worked fine before. Only after the upgrade could he no longer establish a connection. My colleague tried to reconnect the cables and restart the switch ...without success.

Unfortunately, we didn't check LLDP because we had to get the location online quickly.

Tomorrow we want to recreate the whole thing in our lab. As soon as I have more information I will get in touch

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Mon, 29 Jan 2024 10:14:47 GMT

the ports were all active and ready

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Tue, 30 Jan 2024 07:16:51 GMT

Thank you James!

I see in my logs ..

01/30/2024 01:14:04.78 <Info:AAA.LogSsh> Msg from Master : Loaded Private Key of size 1679 from System

01/30/2024 01:14:05.80 <Erro:cm.sys.LoadApplCfgObjFail> "exsshd" application failed to load "exsshdPrivKey" configuration object: Error reading SSH key.

01/30/2024 01:14:05.80 <Info:AAA.LogSsh> Msg from Master : Invalid Private Key used. Please use new key.

01/30/2024 01:14:05.80 <Info:AAA.LogSsh> Msg from Master : Error:Private Key is invalid . Public key generation failed

But it honestly can't be the case that when I update 15 switches I have to make corrections on 8 on site to rebuild the ssh key.

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Tue, 30 Jan 2024 09:09:29 GMT

I found an article that could be our solution to the X690 problem.

Cause

This is a known issue, caused by a Forward Error Correction (FEC) mode being applied during the insertion of 100G optic that then conflicts with the 40G optic.

https://extreme-networks.my.site.com/ExtrArticleDetail?an=000116340&q=X690%20upgrade%20

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

Stefan_K_ — Tue, 30 Jan 2024 14:23:30 GMT

In that case the ports would stay down.

The Error is no longer present in the version you are upgrading to.

If you would have plugged an 100G optic in the past, the 40G optic wouldn't work even before the upgrade.

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Thu, 01 Feb 2024 08:39:12 GMT

That's true, but the switch is connected to an MLAG with two X690s and those are the problem (because of the version 31.7.).

We will now update the one again and do the commands on the MLAG switches .

configure port 57 forward-error-correction on cl74

Hopefully it will work out in the end.

Re: Failed to upgrade X690 and X440-G2 from 31.7.1.4-patch1-77 -> 32.6.1.5-patch1-2

markus_of — Thu, 01 Feb 2024 08:41:14 GMT

This is a solution that we have also considered. 😎