topic RE: Hitless Upgrade on BD8810 in ExtremeSwitching (EXOS/Switch Engine)

Hitless Upgrade on BD8810

Network_Enginee — Wed, 09 Sep 2015 20:35:00 GMT

Good day all! I'm new to Extreme and reading the 15.4 EXOS Concepts Guide I had a question on Hitless Upgrades. I think I completely understand the steps/concepts of Hitless Upgrades except one mention in the documentation:

"If you have a BlackDiamond 8800 series switch and the new ExtremeXOS image supports hitlessupgrade but is not compatible with the current running I/O module image (the I/O version numbers do not match), you cannot perform a hitless upgrade."

Below is the output from my BD 8810. We're upgrading to bd8800-15.5.4.2-patch1-5.xos.

Does that statement in the documentation apply to me? Does that basically say the cards in Slots 1, 2, 4, 8 and 10 in my chassis have to be running same code levels?

A.1 # sh ver

Image : ExtremeXOS version 15.2.3.2 v1523b2-patch1-10 by release-manager
on Wed Jun 5 10:28:14 EDT 2013
BootROM : 1.0.4.5
Diagnostics : 1.19
A.2 # sh switch

SysName: XXX
SysLocation:
SysContact: support@extremenetworks.com, +1 888 257 3000
System MAC: 00:04:96:52:54:21
System Type: BD-8810

SysHealth check: Enabled (Normal)
Recovery Mode: All
System Watchdog: Enabled

Current Time: Wed Sep 9 11:24:23 2015
Timezone: [Auto DST Enabled] GMT Offset: -300 minutes, name is EST.
DST of 60 minutes is currently in effect, name is EDT.
DST begins every second Sunday March at 2:00
DST ends every first Sunday November at 2:00

Boot Time: Thu Jun 26 12:14:48 2014
Boot Count: 10
Next Reboot: None scheduled
System UpTime: 439 days 23 hours 9 minutes 35 seconds

Slot: MSM-A * MSM-B
------------------------ ------------------------
Current State: MASTER BACKUP (In Sync)

Image Selected: secondary secondary
Image Booted: secondary secondary
Primary ver: 12.5.2.6 12.5.2.6
Secondary ver: 15.2.3.2 15.2.3.2
patch1-10 patch1-10

Config Selected: primary.cfg primary.cfg
Config Booted: primary.cfg primary.cfg

primary.cfg Created by ExtremeXOS version 15.2.3.2
774990 bytes saved on Thu Jul 30 14:57:44 2015

Thank you all!!

Garry Frizzell

edit: Serial numbers removed by moderator

RE: Hitless Upgrade on BD8810

Drew_C — Wed, 09 Sep 2015 20:52:00 GMT

Hi Garry,
Based on the serial numbers (that have been removed now), it looks like your chassis consists of the following:

BD 8810 10-Slot Chassis
BD 8800 MSM-48c
BD 8900-10G24X-c
BD 8800 G48Tc

They're all compatible with the version you're upgrading to. In my opinion, performing a hitless upgrade can be more trouble than it's worth. Sometimes it's just easier to reboot the chassis and get it all over with at the same time.

RE: Hitless Upgrade on BD8810

Network_Enginee — Thu, 10 Sep 2015 20:15:00 GMT

Ok, thanks so much for your thoughts Drew! These are datacenter core switches critical to our products being up 24/7 so was trying to do my best to prevent any downtime at all. We do have dual core BD 8810s but the Hitless approach really seemed like a great option to ensure even a fail over (running VRRP too) wouldn't have to happen to the other core Extreme BD.

RE: Hitless Upgrade on BD8810

Drew_C — Thu, 10 Sep 2015 20:15:00 GMT

Generally speaking, it's a lot easier to plan for a full reboot that will very likely come up without issue than it is to plan for a procedure that requires specific criteria and order of execution to go right. With a hitless upgrade, you'll still have to reboot individual I/O modules and each MSM, but you have to plan for that as well. With redundancy in place (like VRRP or MLAG), you can failover from one core to the other and still maintain uptime. You're vulnerable in those short windows while rebooting, in any case.

RE: Hitless Upgrade on BD8810

Paul_Russo — Fri, 11 Sep 2015 00:56:00 GMT

Hey Garry

Normally we only recommend trying a hitless upgrade for versions of code that are minor in their revisions. For example 15.x to 15.y. The reason is that going between major versions may need to change the kernel on the I/O modules.

In my opinion I would not plan on this being a hitless upgrade as you are moving from a 12.5 build to a 15.2 code. If I remember correctly the code should notify you if you can do a hitless upgrade.

I hope that helps

P

RE: Hitless Upgrade on BD8810

Paul_Russo — Fri, 11 Sep 2015 01:02:00 GMT

Sorry Garry I mis-read your versions you are going from to to.

In my testing in the past if the switch allows for the hitless upgrade it has worked well in the past.

Sorry for the confusion from my post.

P

RE: Hitless Upgrade on BD8810

EtherMAN — Fri, 11 Sep 2015 01:30:00 GMT

Quick real world about hitless and the 8900's ... Read in depth as there are some things that will not be 100% hitless. Eaps Mcast come to mind... It is a very good way like the guys said to do patch upgrades inside major revs... 15.1.xx but not going from 15.1 to even 15.2... The line cards do not upgrade till they reboot. If you have card redundancy you can in theory do the msm's and then one card at a time .... Very lengthy process as you can imagine ... In my 18 years using Extreme gear I can count on one hand the number of times we elected to go hitless instead of scheduling a window and having a 3 to 5 minute outage... We have had and still have quite a few chassis and stacks through the years so I agree it is best to reboot if you can... If you have never done a hitless upgrade you best have a spare and practice as it has to be done perfect and in the right sequence... Loosing the console and waiting for it to come back alive is scary so doing a trail run lets you see how long it takes and what to expect.