cancel
Showing results for 
Search instead for 
Did you mean: 

Delta and Complete config update fails

Delta and Complete config update fails

systemscsn
Valued Contributor

 

This started way before summer, and I thought it was fixed.

 

I made a minor change to the power on wifi1 interface (but basically any change i make), select the device, click on Update Device.  Leave the default Update Network Policy and Configuration » Delta Configuration Update checked.  I see the progress bar area show Queued for a few seconds, and then Device Update Failed is displayed next to the AP.

 

Hovering my pointer on the Device Failed to Update, i see this “Could not generate CLI configuration.  Execute Method failed, with class”…………………….

 

 Here are all the steps I tried, all of which failed:

  1. I rebooted the AP, once it came back up, i tried a Delta Configuration Update.  It failed.
  1. I tried a Complete Configuration Update on the AP.  It failed.
  1. Rebooted the AP again, once it came back up I tried a Complete Configuration Update. It failed.
  1. Went into the Aruba 2920 switch, disabled the port that the AP is plugged into (its like unplugging the AP), waited 10 minutes and then re-enabled the port.  Once the AP came back up I tried a Delta Configuration Update.  It failed.
  1. Went back into the Aruba 2920 switch, disabled the same port again, waited 10 minutes and then re-enabled the port.  Once the AP came back up I tried a Complete Configuration Update.  It failed.

 

This is insane.  This issue is going on with any of my 160 AP’s, doesn't matter what change I've made to the AP, whether its changing the channel its on, or the power on one of the interfaces.  I remember when this first reared its ugly head,  we just had the banner show up at the top of the web interface, stating it had been upgraded (IQ Engine?).  We have not changed our network infrastructure in any way, so I'm absolutely positive its nothing we have done.

I need to make small changes to most the AP’s on campus.  I need to fine tune all of our AP’s because of Cross Channel Interference, and so need to make changes with power and channels on the AP’s.  This issue is holding me back.

How can this be resolved?  We have 160 AP’s most of them are attached to the ceilings, so whatever the fix, it better not involve grabbing a paperclip, ladder and interrupting classes.

 

 

Whats going on?  this is an AP650(AH), and all I did this time, was change the power from Auto to 5db.

 

I didnt realize my issue was still going on.  But it never happened until an ExtremeIQ update (i remember seeing a banner ages ago, and thats when this started).  The AP is running 10.0.r5.

 

Any help to get this resolved that does require a paperclip and ladder is appreciated.

 

Thanks,

J.

1 ACCEPTED SOLUTION

systemscsn
Valued Contributor

UPDATE and Possible Solution.

 

Let me start by saying the “fix” I was told to do, was to basically set the AP back to factory.  After that, of course, id have to go back into it and give it a name, its original IP (no im not doing reservations), apply its network policy, and do a complete update.  For 140+ AP’s that is not a very good fix, and quite honestly if other AP’s on the same firmware, plus the same model of AP were not failing, then of course this would work.

The reason I was given for the error, and something i’m very… VERY… upset about, was that when the backend was updated - im thinking multiple times - from Hive to ExtremeIQ, it messed something up on our end because of the version of firmware we are running.  While i do believe it was something done on the backend (we had not made any changes on anything, network, AP, server related on our end) that for reasons i dont know, messed up most of our AP’s so they wouldn't do any kind of update, i dont think im convinced it was our firmware version.  Because if that were true then all of our AP’s on that version would fail, and they are not.  However im mad because if they went into updating the backend knowing it could mess up a customers AP’s, then an email to their customers telling them to upgrade to whatever version would have been appropriate, and i know I never received an email of a possible issue with their backend messing my AP’s up!  To me, there was a basic lack of any kind of Quality Assurance when the backend was updated/migrated from Hive to XIQ and for that someone should take a spanking.

 

While they have escalated my support ticket, i have found a workaround.  For any AP thats failing to Update, and has those errors about Bonjour.  You MUST go into the AP, click on Configure and then select Bonjour Gateway Settings.  Make sure you have the default value of 250 in the Priority Field (some of mine were either blank or had 10 as the value), Check the box for “Override the default realm name”, then Save the Bonjour Settings, Click on Update, Delta and then Update.

That seems to fix the issue.  Of course, you can never uncheck that box, or the error will come back (and it doesn't matter what version of firmware running).

basically its getting that realm name from your Network 360 Plan, if you have created one, and placed the AP on its map.  You wont see the update failed message if you have a value in Priority and dont have the AP within the Network 360 Plan.

 

This whole thing has been a pain, and will continue to be a pain until ive gone through every AP we have and make that change.

 

Once I've gone though that nightmare, i plan on upgrading the firmware to the latest version, even though i can only update them after hours - which is going to take an age to do -.  To think this all came about from wanting to reduce the Transmission Power.  Speaking of which, i did that on an AP that I made the above change to, and while it took the update, wifi0 is still showing auto instead of 5db, which i set it to.  So, who knows, maybe checking the box gets rid of the error message, but doesn't update… really…..  personally i think it just takes an age for the XIQ interface to reflect the changes on the AP, so i’m leaving it alone, and carrying on clicking!

Hope this helps the next poor bugger that may get this issue.

Ill post again if the person who got my escalated support ticket gets back to me with a better, less manual way to fix this problem.

 

Best,

Jason.

 

View solution in original post

52 REPLIES 52

systemscsn
Valued Contributor

OK…. well, i did what we talked about, and got something strange, doesn't make sense and I found a bug.

 

So, I went into that Test AP, clicked on Configure, and changed its policy to the test Policy.  I pushed a Delta Update and it took without issue.

I gave it a few minutes, making sure it really had the test policy, and it did.  I then went back into the device, changed the policy back to the original policy, pushed a delta update and it took WITHOUT issue.

Makes NO SENSE..

I went into the AP (one of the many I've been having issues with), clicked on configure and that was where i noticed something. Under Device Information (after clicking on configure) it shows information in the SNMP Location field. Probably tied into Network 360 Plan.

Thats where i found the bug.  I thought, id just delete the data in the SNMP Location field and hit save - maybe its the location thats causing issue, the SNMP Location is probably tied to the Realm Name -. So i deleted the data in that field, clicked on Save and I get the green banner at the top stating it saved the update successfully.  BUT about 2 seconds later, the screen flashes, comes back and ALL the fields are empty (IP, name, everything) and about another 1-2 seconds later all the fields show back up with the data in them, INCLUDING the SNMP Location field, which id deleted and saved changes!  Thats a bug if ever i saw one.

Of course, that made me think that maybe, just maybe the issue is because i have most of my AP’s assigned locations.  I removed the  troublesome AP from the Network 360 Plan, and tried a delta update.  it failed yet again.

Of course, now what im thinking, is to take one of the AP’s thats giving me grief, applying the test policy to it, do a delta and see if it takes.  Regardless of that outcome, i was then going to go back into the AP and put the original policy on it, and do a Delta update and see what happens..  ideas? thoughts?

 

Hope your morning is going well,

J

UPDATE:  I walked over to where the test AP is (I cant bring it to my office, as I have an AP in the ceiling and I didn't want to create other issues) just to make sure it was offering all three SSID’s which it is.  I noticed a physical difference with the AP.  Its a AP650(AH) just like the ones I'm having issue with. The light on it was solid white.  All our other AP650(AH)’s blink white.  I know, I'm absolutely sure, that the test AP’s LED used to blink white and was not solid white.  Putting it to the test policy and then back to the original policy should mean its back to how it was, like nothing changed.  I have no idea why its LED is now solid white.  Might not be anything, but i found this pretty odd.

 

systemscsn
Valued Contributor

The day just blew up on me!  have to do this first thing tomorrow morning <sigh>.

 

So, if i add the bonjour part, you want me to put in all the same stuff, just with a different name.  I can do that.  What i was going to do, was apply the new policy to the AP, push the update and see if it fails or not.  it probably wont fail.  After that, i was just going to change the policy back to what we are using, and then try pushing the update and see if it fails or not.  Maybe ill do that first.  if it fails, then ill go back, apply the test policy edit it, do the whole bonjour thing with same settings but different name, and then push and see what happens.

 

Make sense?  Sound like a plan?

 

Ill let you know what happens.

 

J

Ash_Finch
Contributor III

Tried replicating with the same firmware, similar bonjour rules, a whole load of comma separated VLANs, tried changing AP locations...but no replication 😞
After you try the network policy without and with the bonjour, if you get the same error try creating another bonjour with the exact same settings, just with a different name. I’m wondering whether the bonjour object has corrupted or something like that. Just a guess, no science behind that though!

systemscsn
Valued Contributor

Good.   Ill go back in, and hit cancel to clear out all the settings, etc. in that template tab. 

The AP’s are on our backbone vlan1 via Ethernet, but based on what SSID a device connects to they will be on a different vlan.  So, if you connect to SSID “A” then you are on vlan 5 and get an IP address that corresponds to that vlan.  You connect to SSID “B” then you are on vlan 10, and same thing, you get an IP address corresponding to that vlan.

 

oh, and here is a screenshot of the BJ GW settings within our default policy:

 

cc66ffdd0270477dae79663160e7fbeb_dae8412e-1fea-4419-bec3-469069e36fa7.png

We basically have to offer that crappy Bonjour protocol all over the place, and I'm sure you know what a bugger that is to do, especially when you have VLANs.  I remember Apple saying they were working on some new type of Bonjour, where it could traverse VLANS, etc. That article died a horrible death about 8+ years ago… probably when they realized its just a crap, chatting, free POS to begin with, and not worth the investment to create one that would actually work and not bog down a network.

I know we could have used the dashes between a lot of those VLANS, but we had a heck of a time getting it to work, and it turned out to be a problem with our DHCP server, which we had to rebuild from scratch (heck of a lot of work… no fun at all).  Then the BG GW started working. 

Ash_Finch
Contributor III

Unless you’re using a custom port type that’s not uplink and Native VLAN1 (default) then I wouldn’t worry about the device template. 

GTM-P2G8KFN