WiNG vx9000 not clustering since being relocated to new HyperV host

[Eric_Burke]

We recently deployed a pair of vx9000’s in HyperV. The controllers are each on a different host. When we started building, we only had one host ready so we built the pair on that box. Now that both hosts are up, we exported and imported one of the units,  Same IP address, just a new host. We found that mistakenly we didn’t clone the MAC and lost the licensing. We then set the MAC back, and confirmed that the vx9000’s coudl still ping eachother. They’re sending hello’s via MINT. A support engineer helped us to realize that the license got dropped, presumably when getting the wrong MAC, so we re-licensed it. Still, we could not get the cluster back up. The engineer dropped the node we’d moved and manually rejoined the cluster. This seemed to work for a brief couple of minutes, but then we kept seeing the active role shifting back and forth between controllers. On the relocated unit, we’re seeing the following message:

CMASTER_CFG_UPDATE_FAIL: Cluster Master Rejecting Config from unknown host. Not sure what else we need to do to resolve the issue?  Any guidance?

[Matt_Unger]

Chris,

Both controllers are currently up and running.  West is the one that has not been modified, East is the one that was built on one server and then migrated to a new physical server.

[ckelly]

Eric,

We recently deployed a pair of vx9000’s in HyperV. The controllers are each on a different host. When we started building, we only had one host ready so we built the pair on that box. Now that both hosts are up, we exported and imported one of the units,  Same IP address, just a new host. We found that mistakenly we didn’t clone the MAC and lost the licensing. We then set the MAC back, and confirmed that the vx9000’s coudl still ping each other.

Can you confirm that from those original 2 VX9000’s that were built that only 1 of them is still running? 

[Matt_Unger]

HI Chris,

Just to confirm all of the AP’s are currently offline.  They are being installed at the new customer site.

[ckelly]

Eric,

There don’t appear to be any APs currently adopted.  Is that correct?  (you can check by running: show adoption status)

I do see on both controllers that they each have one Level-2 neighbor...and I’m guessing that that neighbor is the other controller.

As for the APs and IP addresses - If the APs do have layer-3 access for adopting on their management VLAN, that’s good.  But based on your comment about the trunk on the controller and a tagged VLAN “into the adoption VLAN”...I’m not sure about this. 

Essentially, what I would want to see is the APs using layer-3 (IP address) on whatever VLAN is needed to be able to adopt to the controllers.

(it may come down, if the culprit isn’t found fairly soon, to taking a look at the full configs from each controller.  Sometimes a misconfiguration in a place that I”m assuming is correct can be hiding out somewhere and cause you to end up chasing your tail)

Again, for now though...fix the controller cluster members entries area but you need to use the correct MINT routing level.  If in doubt, use Level-2.  But this would also mean that you need to ensure that the APs are also adopting using MINT Level-2.

[Eric_Burke]

@Chris Kelly , the AP’s do have IP’s in that VLAN, but as I understand it, they’re not really necessary. They get a DHCP address in the same VLAN that the controller has a subinterface. The vx9000 has a single GigE trunk, untagged into our management vlan and tagged into the adption vlan. The controllers can reach one another over the management network via their local default gateway (the firewall).