Serious problem with MESH on wing ap410, 7.5.1.1-020R

[Tomasz_Lubas]

Hello, maybe someone can share experience or give advice about MESH configuraion. 

I have implemented mesh for warehouse, three alleys, 6 root AP’s with panel antennas and two mobile AP’s on the gantrys.

The problem is - after 4 days working, both mobile APs disconnected from mesh and unadopted from controller. After some time AP’s back and working for some time then again, both disapeared. 

my config is:

meshpoint-qos-policy MESH

meshpoint MESH
meshid 1
beacon-format mesh-point
control-vlan 1
allowed-vlans 1-4094
use meshpoint-qos-policy MESH
security-mode psk
wpa2 psk 0 key
root
!

profile ap410 mesh-root
no autoinstall configuration
no autoinstall firmware
interface radio1
  shutdown
interface radio2
  channel 44ww
  rts-threshold 1
  meshpoint MESH bss 1
  antenna-gain 6.0
  no dynamic-chain-selection
interface radio3
  shutdown
interface bluetooth1
  shutdown
  mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
rf-domain-manager capable
rf-domain-manager priority 1
controller vlan 1
service pm sys-restart
router ospf
meshpoint-device MESH
  root
  path-method uniform
adoption-mode controller
!
profile ap410 non-root-MESH
no autoinstall configuration
no autoinstall firmware
interface radio1
  shutdown
interface radio2
  channel 44ww
  rts-threshold 1
  meshpoint MESH bss 1
  antenna-gain 6.0
  no dynamic-chain-selection
interface radio3
  shutdown
interface bluetooth1
  shutdown
  mode le-sensor
interface ge1
interface ge2
interface pppoe1
use firewall-policy default
service pm sys-restart
router ospf
meshpoint-device MESH
  no root
  path-method mobile-snr-leaf
adoption-mode controller
!
rf-domain default
timezone CET
country-code pl
ad-wips-wireless-mitigation disable
ad-wips-wired-mitigation disable
!

ap410 48-9B-D5-FE-1B-F2
 use profile mesh-root
 use rf-domain default
 hostname CAP-AK-FE1BF2
 interface radio1
  ldpc
 interface radio2
  ldpc
 interface vlan1
  description "Virtual Interface for LAN by Wizard"
  ip address 172.28.0.213/24
  ip address zeroconf secondary
  ip dhcp client request options all
 ntp server 172.28.0.2 
 virtual-controller
 rf-domain-manager capable
 rf-domain-manager priority 255
 no adoption-mode
!

ap410 48-9B-D5-FE-2C-18
use profile mesh-root
use rf-domain default
hostname ap410-FE2C18
interface vlan1
  ip address 172.28.0.211/24
!
ap410 48-9B-D5-FE-2C-54
use profile non-root-MESH
use rf-domain default
hostname ap410-FE2C54
interface vlan1
  ip address 172.28.0.215/24
!

events from controller:

2021-01-24 00:54:21     ap410-FE2D0E  MESH       MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:21     ap410-FE2D0E  MESH       MESHPOINT_UP         Meshpoint 'MESH' detected up
2021-01-24 00:54:12     ap410-FE2C54  MESH       MESHPOINT_LOOP_PREVENT_OFF Meshpoint loop prevention off (port ALL), all wired traffic is allowed
2021-01-24 00:54:12     ap410-FE2C54  MESH       MESHPOINT_UP         Meshpoint 'MESH' detected up
2021-01-24 00:54:01     ap410-FE2C54  SYSTEM     CONFIG_REVISION      Configuration revision updated to 37 from 38
2021-01-24 00:54:01     ap410-FE2C54  SYSTEM     CONFIG_COMMIT        Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:01     ap410-FE2C54  DEVICE     ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:02     CAP-AK-FE1BF2  DEVICE     DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2C-54 ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:02     CAP-AK-FE1BF2  DEVICE     ADOPTED              Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|
2021-01-24 00:54:00     ap410-FE2D0E  SYSTEM     CONFIG_REVISION      Configuration revision updated to 37 from 38
2021-01-24 00:54:00     ap410-FE2D0E  SYSTEM     CONFIG_COMMIT        Configuration commit by user 'cfgd' (update own config) from '127.0.0.1'
2021-01-24 00:54:00     ap410-FE2D0E  DEVICE     ADOPTED_TO_CONTROLLER Joined successfully with controller 'CAP-AK-FE1BF2'(55.FE.1B.F2)
2021-01-24 00:54:00     CAP-AK-FE1BF2  DEVICE     DEVICE_AUTOUP_NO_NEED AUTOUPGRADE: ap410 mac 48-9B-D5-FE-2D-0E ver 7.5.1.1-020R Autoupgrade not required or not available
2021-01-24 00:54:00     CAP-AK-FE1BF2  DEVICE     ADOPTED              Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' adopted and configured. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2D-0E(ap410-FE2D0E) is offline, last seen:10 minutes ago on switchport -
2021-01-23 12:07:00     CAP-AK-FE1BF2  DEVICE     OFFLINE              Device 48-9B-D5-FE-2C-54(ap410-FE2C54) is offline, last seen:10 minutes ago on switchport -
2021-01-23 11:56:56     CAP-AK-FE1BF2  DEVICE     UNADOPTED            Device('ap410-FE2D0E'/'ap410'/48-9B-D5-FE-2D-0E) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-70-20|48-9B-D5-F3-70-30|48-9B-D5-FE-2D-13|
2021-01-23 11:56:46     CAP-AK-FE1BF2  DEVICE     UNADOPTED            Device('ap410-FE2C54'/'ap410'/48-9B-D5-FE-2C-54) at rf-domain:'default' unadopted. Radios: Count=3, Bss: 48-9B-D5-F3-6C-40|48-9B-D5-F3-6C-50|48-9B-D5-FE-2C-59|

There was site survey made - signal coverage was ok.

any ideas? Why both ap at the same time disconnects ?

regards,

Tomek

[Tomasz_Lubas]

Hi,

Customer expecting fast path change after one root ap failure, less than 1-2 sec. For now it is 2-14 sec.

Also sometimes when mobile ap changing root, there is 1-2 sec outage, we expected fast roaming less than 50ms.

Problem is that mobile gantry, connected via MESH mobile AP is connection sensive. It communicate to control device via udp multicast messages. If there is no communication when it requesting orders then it stops. 

regards,

Tomek

[Valter_Veiga]

Hi Tomasz

Failure scenarios depend of the check/verification timeout. What was your expectation time ?

[Tomasz_Lubas]

Valter,

after downgrade it looks more stable but i’ll give it some time to check if it is permanent. Last time it was working 4 days till crash.

Have you tested failure scenarios eg. disabling radio on root AP mobile is attached to ?

In mine enviromene it last 2-9 sec to reattach.

could you share your config ?

regards,

Tomek

[Tomasz_Lubas]

Hello Valter, thank you for downgrade sugestion. I had 7.5.1.1 then 7.5.1.3. 

It is very unstable and unpredictable. Root ap losing neighbours without reason. 
Mobile APs losing connections and doesn't attach to mesh. 

I’ll try downgrade and let you know.

regards,

Tomek