ExtremeWireless (WiNG)

Hi
over the last couple of day the RFS7K has dropped off the network and stops all MU form connecting. The WIF is still visable ( Ap's are 7131 & 7532 ) but no connection can be made.
I have looked on the Primary unit and there was some crash files about 6 in total but dating from earlier in the this month.

But its all a bit meaningless to me:

Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:964 server_tasklet() session args = {'xfer_args': {'tzsp_encap': True}, 'output_type': 9}
Return success for start_session

Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:1034 session_initiator_tasklet() requesting host 70.81.BE.8E
Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/client.py:778 req_listen_tasklet() Peer 70.81.BE.8E connected over seqpacket port 36

Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:1066 session_initiator_tasklet() notified 1 of 1 hosts

In handle_request_tasklet =====> req message

Hello Phil,

It seems as you said, a soft lockup, caused by the dataplane process. Could you please give us a better view of how are those APs deployed? Different domains and level 2 MiNT links? Can you please share the output of the following commands on the panicking device?

#show mint links
#service show top
#show cluster members detail

How often is that panic happening?

Thanks in advance.

link ip-172.17.146.105:24576 at level 1, 1 adjacencies
link ip-172.17.146.106:24576 at level 1, 0 adjacencies
link vlan-1 at level 1, 15 adjacencies, DIS 70.38.0A.F9 (self)

PID STATUS RSS PPID %CPU %MEM COMMAND
1061 R 334641052 1.9 15.0 cfgd
1132 S 331441122 0.1 14.8 ssm
1060 S 9680 1042 0.0 4.3 rim
12854 S 8860 12842 0.0 3.9 mapsh
1167 S 7156 1157 0.0 3.2 hsd
1187 S 6984 1179 0.0 3.1 dpd2
1449 S 4736 1441 0.0 2.1 snmpd
1514 S 3640 1492 0.0 1.6 lighttpd
1104 S 3140 1096 4.3 1.4 nsm
1029 S 2756 1021 0.0 1.2 logd

---------------------------------------------------------------------------------------------------------
ID MAC MODE AP COUNT AAP COUNT AP LICENSE AAP LICENSE VERSION
---------------------------------------------------------------------------------------------------------
70.38.0A.F9 00-15-70-38-0A-F9 Standby 0 0 0 0 5.8.5.0-016R
70.81.BE.8E 00-15-70-81-BE-8E Active 0 15 128 0 5.8.5.0-016R

Hi Phil,

Is the device 70.38.0A.F9 acting as well as cluster master? You can check it with the following command:

#show cluster members

Hi
No, 70:38:0A:F0 is in standby

I don't think your crashes are related to your problems and besides the panic which was seen during taking of packet capture most porbably - the rest are not crashes or you are just not putting crash locations. And the logs above are from 6521 - not the Aps you have problems with.
You need to have GTAC look at your config and check if config is correct.

Hi
I did no packet capture, the painc wa in the crash file location on the RFS,
What are these messages that are repeated ?
<6>Pretty Call Trace:
<6> SP 0x88171a80, RA 0x8010b4dc, PC 0x80109544 dump_stack+0x0/0x1c
<6>250: addiu sp,sp,-152
<6>346: frame_size 152
<6> SP 0x88171b18, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
<6>250: addiu sp,sp,-24
<6>346: frame_size 24
<6> SP 0x88171b30, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
<6>250: addiu sp,sp,-24
<6>346: frame_size 24
<6> SP 0x88171b48, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
<6>250: addiu sp,sp,-24
<6>346: frame_size 24
<6> SP 0x88171b60, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
<6>250: addiu sp,sp,-24
<6>346: frame_size 24
<6> SP 0x88171b78, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c

This is just stack fuction call. Is this panic hapenning constantly - or you just have one time panic occurance? You can look at the timestamp and see how long ago panic occured. Again - you said MUs are not connecting and unless your controller is constantly rebooting - that panic is not what's causing you a problem. Probably something else in the config.

Hi again Phil,

Sorry for the confusion. What I meant is, if the controller with MiNT ID 70:38:0A:F0 has the master status on the cluster showing up as true. Can you please share the output of the following command?

#show cluster members

rfs7000-Backup 70.38.0A.F9 00-15-70-38-0A-F9 False standby 00:01:01 ago
rfs7000-Primary 70.81.BE.8E 00-15-70-81-BE-8E True active self

rfs7000-Backup 70.38.0A.F9 00-15-70-38-0A-F9 False standby 00:01:01 ago
rfs7000-Primary 70.81.BE.8E 00-15-70-81-BE-8E True active self