Serious problem with MESH on wing ap410, 7.5.1.1-020R

[Tomasz_Lubas]

Hello, maybe someone can share experience or give advice about MESH configuraion. 

I have implemented mesh for warehouse, three alleys, 6 root AP’s with panel antennas and two mobile AP’s on the gantrys.

The problem is - after 4 days working, both mobile APs disconnected from mesh and unadopted from controller. After some time AP’s back and working for some time then again, both disapeared. 

my config is:

meshpoint-qos-policy MESH

meshpoint MESH
meshid 1
beacon-format mesh-point
control-vlan 1
allowed-vlans 1-4094
use meshpoint-qos-policy MESH
security-mode psk
wpa2 psk 0 key
root
!

profile ap410 mesh-root
no autoinstall configuration
no autoinstall firmware
interface radio1
  shutdown
interface radio2
  channel 44ww
  rts-threshold 1
  meshpoint MESH bss 1
  antenna-gain 6.0
  no dynamic-chain-selection
interface radio3
  shutdown
interface bluetooth1
  shutdown
  mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
rf-domain-manager capable
rf-domain-manager priority 1
controller vlan 1
service pm sys-restart
router ospf
meshpoint-device MESH
  root
  path-method uniform
adoption-mode controller
!
profile ap410 non-root-MESH
no autoinstall configuration
no autoinstall firmware
interface radio1
  shutdown
interface radio2
  channel 44ww
  rts-threshold 1
  meshpoint MESH bss 1
  antenna-gain 6.0
  no dynamic-chain-selection
interface radio3
  shutdown
interface bluetooth1
  shutdown
  mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
service pm sys-restart
router ospf
meshpoint-device MESH
  no root
  path-method mobile-snr-leaf
adoption-mode controller
!
rf-domain default
timezone CET
country-code pl
ad-wips-wireless-mitigation disable
ad-wips-wired-mitigation disable
!

ap410 48-9B-D5-FE-1B-F2
 use profile mesh-root
 use rf-domain default
 hostname CAP-AK-FE1BF2
 interface radio1
  ldpc
 interface radio2
  ldpc
 interface vlan1
  description "Virtual Interface for LAN by Wizard"
  ip address 172.28.0.213/24
  ip address zeroconf secondary
  ip dhcp client request options all
 ntp server 172.28.0.2 
 virtual-controller
 rf-domain-manager capable
 rf-domain-manager priority 255
 no adoption-mode
!

ap410 48-9B-D5-FE-2C-18
use profile mesh-root
use rf-domain default
hostname ap410-FE2C18
interface vlan1
  ip address 172.28.0.211/24
!
ap410 48-9B-D5-FE-2C-54
use profile non-root-MESH
use rf-domain default
hostname ap410-FE2C54
interface vlan1
  ip address 172.28.0.215/24
!

events from controller:

2021-01-24 00:54:21     ap410-FE2D0E  MESH       MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:21     ap410-FE2D0E  MESH       MESHPOINT_UP         Meshpoint 'MESH' detected up
2021-01-24 00:54:12     ap410-FE2C54  MESH       MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:12     ap410-FE2C54  MESH       MESHPOINT_UP         Meshpoint 'MESH' detected up
2021-01-24 00:54:01     ap410-FE2C54  SYSTEM     CONFIG_REVISION      Configuration revision updated to 37 from 38
2021-01-24 00:54:01     ap410-FE2C54  SYSTEM     CONFIG_COMMIT        Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:01     ap410-FE2C54  DEVICE     ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:02     CAP-AK-FE1BF2  DEVICE     DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2C-54 ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:02     CAP-AK-FE1BF2  DEVICE     ADOPTED              Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|
2021-01-24 00:54:00     ap410-FE2D0E  SYSTEM     CONFIG_REVISION      Configuration revision updated to 37 from 38
2021-01-24 00:54:00     ap410-FE2D0E  SYSTEM     CONFIG_COMMIT        Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:00     ap410-FE2D0E  DEVICE     ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:00     CAP-AK-FE1BF2  DEVICE     DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2D-0E ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:00     CAP-AK-FE1BF2  DEVICE     ADOPTED              Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2D-0E(ap410-FE2D0E) is offline, last seen:10 minutes ago on switchport -
2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2C-54(ap410-FE2C54) is offline, last seen:10 minutes ago on switchport -
2021-01-23 11:56:56     CAP-AK-FE1BF2  DEVICE     UNADOPTED            Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 11:56:46     CAP-AK-FE1BF2  DEVICE     UNADOPTED            Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|

There was site survey made - signal coverage was ok.

any ideas? Why both ap at the same time disconnects ?

regards,

Tomek

[Valter_Veiga]

Hi Tomasz 

I have issues with 7.5.1.2 version, i had to downgrad to 7.4.1.4-004R, I have post:

MESH firmware 7.5.1.2-008R 7662 and 7632 | Extreme Networks Support Community

Have you run the MESH in platform  AP410 with the 7.5.1.2 ?

[Tomasz_Lubas]

Hello Ovais Qayyum,

thank you very much for your answer and good advice. We changed channels width from 80 to 20 MHz.

After analysys i saw that one of root AP has no mesh neighbours … after restart both mobile ap readopted to controller again and we check if it was the only problem.

regards,

Tomek

[Ovais_Qayyum]

Hi Tomek,

The VMM support for AP410/460 was officially added in release 7.5.1.2-008R. So, I would recommend you upgrade the APs, that should help.

My other recommendation would be to NOT use 80MHz channels for VMM/Mobile Mesh use case, it’s not a good idea and will always create issues. You have 80MHz channel 44ww selected under the profile ap410 mesh-root which sets all 6 of your root APs to the same 80MHz channel, this will cause channel utilization, transmit retries, and neighbor interference issues. Since you are also not using Smart-RF, there won’t be any mitigation and your Root APs will compete with each other pretty badly. As you mentioned, the APs are fitted with panel antennas, ensuring coverage overlap and proper RSSI threshold on an 80MHz channel would be a challenge.  

For WiNG MCX VMM use cases, we recommend using 20MHz channels to take advantage of better receive sensitivity (Rx sensitivity for 80MHz channel is going to be poor) which will help in roaming and handoffs when the VMM moves from one AP to another. You can set your Root APs to say 2-3 different 20MHz channels and add those channels in the VMM scan list, every time VMM will move it will scan those channels and pick the best one. 

In addition, I am seeing the following logs on VMM AP; it very much looks like a radar CAC event where AP must back off for at least 2 minutes in event of a radar hit (it can go as long as 10 minutes in some cases). 

2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2D-0E(ap410-FE2D0E) is offline, last seen:10 minutes ago on switchport -
2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2C-54(ap410-FE2C54) is offline, last seen:10 minutes ago on switchport -      

To confirm if the AP had a radar hit, use the following command:

VX9000-Primary~#service show wireless radar-status

if these suggestions don't help, we will run an MCX debug to find out the issue. 

Regards,

Ovais