Upgrading firmware on C5210 availability pair (order of install)

  • 0
  • 1
  • Question
  • Updated 11 months ago
  • Answered
I have two C5210 controllers in an availability pair. I need to perform an upgrade and this will be the first one I have done since the pair went up.

According to an Extreme KB ...
In a High Availability pair when upgrading the Extreme Wireless Controller Local AP's will fail over to the backup controller and show as Foreign. In order to upgrade the AP's they must be manually released back to the Primary/Local controller.  Check to see AP upgrade behavior settings under the AP menu/ Global  Controlled upgrade to manually upgrade AP's  Option 2 Always upgrade AP to default image (overrides Controlled Upgrade settings)  
I normally do this at 2:00AM and sleep through it! But if I manually have to move the AP's back to the other controller - that is a no go.

Can I do this ??
  1. Upgrade the secondary controller right now (which presently has zero local AP's, and 100% foreign).
  2. Download and *schedule* the upgrade for the primary controller at 2:00AM.
  3. When the AP's try to fly over to the secondary controller, they will be forced into an upgrade.
Or - is there a preferred order of doing this?
Photo of Steve Ballantyne

Steve Ballantyne

  • 5,566 Points 5k badge 2x thumb

Posted 11 months ago

  • 0
  • 1
Photo of Craig Guilmette

Craig Guilmette, Employee

  • 2,440 Points 2k badge 2x thumb
Hello Steve
No that will not work as the AP's will only upgrade when they have been manually rebooted when connected to their local controller. The only thing I can think of is to upgrade the foreign controller then shut it off. Then schedule the local controller for some time at night when nobody is using the wifi and when the controller goes down and comes back up from the upgrade it will then automatically upgrade all the AP's. BUT there will be an outage for wifi users. The only other option would be to schedule AP maintenance after the controller has been upgraded to reboot the AP's during the night doing the controllers during the day 1 at a time to prevent any outage. 
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 45,306 Points 20k badge 2x thumb
The right order would be...

- upgrade controller#1 now, APs will failover
- schedule upgrade of controller#2 at 2am = the APs will failover to controller#1 and upgrade (if you don't use fastfailover).

The reboot of the AP on the local controller is AFAIK only required if fastfailover is enabled.
(Edited)
Photo of Craig Guilmette

Craig Guilmette, Employee

  • 2,440 Points 2k badge 2x thumb
True but 99% of our customers use fastfailover. 
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 45,306 Points 20k badge 2x thumb
And I'm sure 99% of them have no idea how it works and only enabled it because of the cool name :-0
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
1.  Fail ALL of the Local APs to the other controller by selecting the check box for ALL local APs and choosing "Release".  
2.  Upgrade controller number 1
3.  Fail all the APs back once its back up, including all the APs from the other controller
4. Upgrade other controller
5. Fail foreign APs back to other controller once it reboots
6. Under global setting under APs, create a maintenance cycle to reboot the APs from 2AM with a 2 hour duration.  
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 45,306 Points 20k badge 2x thumb
#1 will only work if fastfailover is enabled,
That is the big question, which mode is used as that makes a big difference
Photo of Steve Ballantyne

Steve Ballantyne

  • 5,566 Points 5k badge 2x thumb
Thanks for the advice guys. I *do* have Fast Failover enabled at the moment. But with the problems I have experienced with the present version of firmware, it's not really doing the trick.

Perhaps I should disable Fast Failover to make this upgrade smoother. Then re-enable it after everything is up and running again.
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
I don't think you will have issues upgrading.  I honestly have NEVER had a botched upgrade.
Photo of Steve Ballantyne

Steve Ballantyne

  • 5,566 Points 5k badge 2x thumb
Just completed the upgrade. I decided to do it late, but not 2:00AM late.  ;-) Didn't have any issues.

The method I took was:
  1. Disabled Fast Failover on both controllers.
  2. Upgraded Controller #1 (all AP's immediately jumped to controller #2).
  3. Waited for Controller #1 upgrade to complete.
  4. Upgraded Controller #2 (all AP's jumped to controller #1)
  5. Watched all the AP's upgrade off of Controller #1, and waited for Controller #2 to come back.
  6. Done!
The only issue I have ever had with upgrades is with my AP's at remote sites where I am doing a B@AP and also a B@C. Reason being - I want the guest network there to use the local cable modem bandwidth. But in doing so, the upgrades fail because they cannot TFTP the new firmware down through a "NAT connection" (over VPN). Something about the fact that it's connecting to the controller using a public IP address. But then skipping to the internal IP address and failing when it comes to TFTP the file down.

Anyway, the fix I have been using is to pre-upgrade the firmware manually on each of those AP's (and then not reboot them). I wrote a series of shell scripts which work pretty good. To be honest, I am not sure that this is required any more. But I am not going to skip this step to find out and have to drive all over the State in the wee-hours of the morning to fix it.
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 45,306 Points 20k badge 2x thumb
Welcome to the 1% ;-)

Regarding the APs that are connected via NAT, you are correct, software upgrade via that link still doesn't work.

I've done some troubleshooting long time ago and the issue is that the AP doesn't use the tunnel to connect to the controller for the software upgrade.
Instead a new TFTP connection is setup and unfortunately the internal IP is used instead of the internet address which isn't reachable from the remote site.
 
As that isn't solved till now I assume it's very deep in the code and not easy to fix.