topic Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R in ExtremeWireless (WiNG)

Serious problem with MESH on wing ap410, 7.5.1.1-020R

Tomasz_Lubas — Tue, 26 Jan 2021 00:13:23 GMT

Hello, maybe someone can share experience or give advice about MESH configuraion.

I have implemented mesh for warehouse, three alleys, 6 root AP’s with panel antennas and two mobile AP’s on the gantrys.

The problem is - after 4 days working, both mobile APs disconnected from mesh and unadopted from controller. After some time AP’s back and working for some time then again, both disapeared.

my config is:

meshpoint-qos-policy MESH

meshpoint MESH
meshid 1
beacon-format mesh-point
control-vlan 1
allowed-vlans 1-4094
use meshpoint-qos-policy MESH
security-mode psk
wpa2 psk 0 key
root
!

profile ap410 mesh-root
no autoinstall configuration
no autoinstall firmware
interface radio1
shutdown
interface radio2
channel 44ww
rts-threshold 1
meshpoint MESH bss 1
antenna-gain 6.0
no dynamic-chain-selection
interface radio3
shutdown
interface bluetooth1
shutdown
mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
rf-domain-manager capable
rf-domain-manager priority 1
controller vlan 1
service pm sys-restart
router ospf
meshpoint-device MESH
root
path-method uniform
adoption-mode controller
!
profile ap410 non-root-MESH
no autoinstall configuration
no autoinstall firmware
interface radio1
shutdown
interface radio2
channel 44ww
rts-threshold 1
meshpoint MESH bss 1
antenna-gain 6.0
no dynamic-chain-selection
interface radio3
shutdown
interface bluetooth1
shutdown
mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
service pm sys-restart
router ospf
meshpoint-device MESH
no root
path-method mobile-snr-leaf
adoption-mode controller
!
rf-domain default
timezone CET
country-code pl
ad-wips-wireless-mitigation disable
ad-wips-wired-mitigation disable
!

ap410 48-9B-D5-FE-1B-F2
use profile mesh-root
use rf-domain default
hostname CAP-AK-FE1BF2
interface radio1
ldpc
interface radio2
ldpc
interface vlan1
description "Virtual Interface for LAN by Wizard"
ip address 172.28.0.213/24
ip address zeroconf secondary
ip dhcp client request options all
ntp server 172.28.0.2
virtual-controller
rf-domain-manager capable
rf-domain-manager priority 255
no adoption-mode
!

ap410 48-9B-D5-FE-2C-18
use profile mesh-root
use rf-domain default
hostname ap410-FE2C18
interface vlan1
ip address 172.28.0.211/24
!
ap410 48-9B-D5-FE-2C-54
use profile non-root-MESH
use rf-domain default
hostname ap410-FE2C54
interface vlan1
ip address 172.28.0.215/24
!

events from controller:

2021-01-24 00:54:21 ap410-FE2D0E MESH MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:21 ap410-FE2D0E MESH MESHPOINT_UP Meshpoint 'MESH' detected up
2021-01-24 00:54:12 ap410-FE2C54 MESH MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:12 ap410-FE2C54 MESH MESHPOINT_UP Meshpoint 'MESH' detected up
2021-01-24 00:54:01 ap410-FE2C54 SYSTEM CONFIG_REVISION Configuration revision updated to 37 from 38
2021-01-24 00:54:01 ap410-FE2C54 SYSTEM CONFIG_COMMIT Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:01 ap410-FE2C54 DEVICE ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:02 CAP-AK-FE1BF2 DEVICE DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2C-54 ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:02 CAP-AK-FE1BF2 DEVICE ADOPTED Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|
2021-01-24 00:54:00 ap410-FE2D0E SYSTEM CONFIG_REVISION Configuration revision updated to 37 from 38
2021-01-24 00:54:00 ap410-FE2D0E SYSTEM CONFIG_COMMIT Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:00 ap410-FE2D0E DEVICE ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:00 CAP-AK-FE1BF2 DEVICE DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2D-0E ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:00 CAP-AK-FE1BF2 DEVICE ADOPTED Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 12:07:00 CAP-AK-FE1BF2 DEVICE OFFLINE Device 48-9B-D5-FE-2D-0E(ap410-FE2D0E) is offline, last seen:10 minutes ago on switchport -
2021-01-23 12:07:00 CAP-AK-FE1BF2 DEVICE OFFLINE Device 48-9B-D5-FE-2C-54(ap410-FE2C54) is offline, last seen:10 minutes ago on switchport -
2021-01-23 11:56:56 CAP-AK-FE1BF2 DEVICE UNADOPTED Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 11:56:46 CAP-AK-FE1BF2 DEVICE UNADOPTED Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|

There was site survey made - signal coverage was ok.

any ideas? Why both ap at the same time disconnects ?

regards,

Tomek

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Ovais_Qayyum — Tue, 26 Jan 2021 08:15:40 GMT

Hi Tomek,

The VMM support for AP410/460 was officially added in release 7.5.1.2-008R. So, I would recommend you upgrade the APs, that should help.

My other recommendation would be to NOT use 80MHz channels for VMM/Mobile Mesh use case, it’s not a good idea and will always create issues. You have 80MHz channel 44ww selected under the profile ap410 mesh-root which sets all 6 of your root APs to the same 80MHz channel, this will cause channel utilization, transmit retries, and neighbor interference issues. Since you are also not using Smart-RF, there won’t be any mitigation and your Root APs will compete with each other pretty badly. As you mentioned, the APs are fitted with panel antennas, ensuring coverage overlap and proper RSSI threshold on an 80MHz channel would be a challenge.

For WiNG MCX VMM use cases, we recommend using 20MHz channels to take advantage of better receive sensitivity (Rx sensitivity for 80MHz channel is going to be poor) which will help in roaming and handoffs when the VMM moves from one AP to another. You can set your Root APs to say 2-3 different 20MHz channels and add those channels in the VMM scan list, every time VMM will move it will scan those channels and pick the best one.

In addition, I am seeing the following logs on VMM AP; it very much looks like a radar CAC event where AP must back off for at least 2 minutes in event of a radar hit (it can go as long as 10 minutes in some cases).

2021-01-23 12:07:00 CAP-AK-FE1BF2 DEVICE OFFLINE Device 48-9B-D5-FE-2D-0E(ap410-FE2D0E) is offline, last seen:10 minutes ago on switchport -
2021-01-23 12:07:00 CAP-AK-FE1BF2 DEVICE OFFLINE Device 48-9B-D5-FE-2C-54(ap410-FE2C54) is offline, last seen:10 minutes ago on switchport -

To confirm if the AP had a radar hit, use the following command:

VX9000-Primary~#service show wireless radar-status

if these suggestions don't help, we will run an MCX debug to find out the issue.

Regards,

Ovais

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Tomasz_Lubas — Tue, 26 Jan 2021 18:26:28 GMT

Hello Ovais Qayyum,

thank you very much for your answer and good advice. We changed channels width from 80 to 20 MHz.

After analysys i saw that one of root AP has no mesh neighbours … after restart both mobile ap readopted to controller again and we check if it was the only problem.

regards,

Tomek

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Valter_Veiga — Wed, 27 Jan 2021 16:37:10 GMT

Hi Tomasz

I have issues with 7.5.1.2 version, i had to downgrad to 7.4.1.4-004R, I have post:

MESH firmware 7.5.1.2-008R 7662 and 7632 | Extreme Networks Support Community

Have you run the MESH in platform AP410 with the 7.5.1.2 ?

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Tomasz_Lubas — Mon, 01 Feb 2021 16:57:59 GMT

Hello Valter, thank you for downgrade sugestion. I had 7.5.1.1 then 7.5.1.3.

It is very unstable and unpredictable. Root ap losing neighbours without reason.
Mobile APs losing connections and doesn't attach to mesh.

I’ll try downgrade and let you know.

regards,

Tomek

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Tomasz_Lubas — Mon, 01 Feb 2021 23:12:14 GMT

Valter,

after downgrade it looks more stable but i’ll give it some time to check if it is permanent. Last time it was working 4 days till crash.

Have you tested failure scenarios eg. disabling radio on root AP mobile is attached to ?

In mine enviromene it last 2-9 sec to reattach.

could you share your config ?

regards,

Tomek

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Valter_Veiga — Thu, 04 Feb 2021 21:59:04 GMT

Hi Tomasz

Failure scenarios depend of the check/verification timeout. What was your expectation time ?

Re: Serious problem with MESH on wing ap410, 7.5.1.1-020R

Tomasz_Lubas — Fri, 05 Feb 2021 20:53:55 GMT

Hi,

Customer expecting fast path change after one root ap failure, less than 1-2 sec. For now it is 2-14 sec.

Also sometimes when mobile ap changing root, there is 1-2 sec outage, we expected fast roaming less than 50ms.

Problem is that mobile gantry, connected via MESH mobile AP is connection sensive. It communicate to control device via udp multicast messages. If there is no communication when it requesting orders then it stops.

regards,

Tomek