ExtremeWireless (WiNG)

Expand all | Collapse all

RFS7000 Crashing - Wing 5.8.5

  • 1.  RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 06:43
    Hi
    over the last couple of day the RFS7K has dropped off the network and stops all MU form connecting. The WIF is still visable ( Ap's are 7131 & 7532 ) but no connection can be made.
    I have looked on the Primary unit and there was some crash files about 6 in total but dating from earlier in the this month.

    But its all a bit meaningless to me:

    Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:964 server_tasklet() session args = {'xfer_args': {'tzsp_encap': True}, 'output_type': 9}
    Return success for start_session

    Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:1034 session_initiator_tasklet() requesting host 70.81.BE.8E
    Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/client.py:778 req_listen_tasklet() Peer 70.81.BE.8E connected over seqpacket port 36

    Thu Sep 7 10:39:35 2017: /home/spin/workspace/ncap_br2/build_br2/obj/qs5/src/sys/cfgd/remote_debug/server.py:1066 session_initiator_tasklet() notified 1 of 1 hosts

    In handle_request_tasklet =====> req message


  • 2.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 09:53
    Hello Phil,

    It seems as you said, a soft lockup, caused by the dataplane process. Could you please give us a better view of how are those APs deployed? Different domains and level 2 MiNT links? Can you please share the output of the following commands on the panicking device?

    #show mint links
    #service show top
    #show cluster members detail

    How often is that panic happening?

    Thanks in advance.



  • 3.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 10:31
    link ip-172.17.146.105:24576 at level 1, 1 adjacencies
    link ip-172.17.146.106:24576 at level 1, 0 adjacencies
    link vlan-1 at level 1, 15 adjacencies, DIS 70.38.0A.F9 (self)

    PID STATUS RSS PPID %CPU %MEM COMMAND
    1061 R 334641052 1.9 15.0 cfgd
    1132 S 331441122 0.1 14.8 ssm
    1060 S 9680 1042 0.0 4.3 rim
    12854 S 8860 12842 0.0 3.9 mapsh
    1167 S 7156 1157 0.0 3.2 hsd
    1187 S 6984 1179 0.0 3.1 dpd2
    1449 S 4736 1441 0.0 2.1 snmpd
    1514 S 3640 1492 0.0 1.6 lighttpd
    1104 S 3140 1096 4.3 1.4 nsm
    1029 S 2756 1021 0.0 1.2 logd

    ---------------------------------------------------------------------------------------------------------
    ID MAC MODE AP COUNT AAP COUNT AP LICENSE AAP LICENSE VERSION
    ---------------------------------------------------------------------------------------------------------
    70.38.0A.F9 00-15-70-38-0A-F9 Standby 0 0 0 0 5.8.5.0-016R
    70.81.BE.8E 00-15-70-81-BE-8E Active 0 15 128 0 5.8.5.0-016R



  • 4.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 10:57
    Hi Phil,

    Is the device 70.38.0A.F9 acting as well as cluster master? You can check it with the following command:

    #show cluster members


  • 5.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 12:10
    Hi
    No, 70:38:0A:F0 is in standby



  • 6.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-22-2017 01:19
    I don't think your crashes are related to your problems and besides the panic which was seen during taking of packet capture most porbably - the rest are not crashes or you are just not putting crash locations. And the logs above are from 6521 - not the Aps you have problems with.
    You need to have GTAC look at your config and check if config is correct.


  • 7.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-22-2017 04:04
    Hi
    I did no packet capture, the painc wa in the crash file location on the RFS,
    What are these messages that are repeated ?
    <6>Pretty Call Trace:
    <6> SP 0x88171a80, RA 0x8010b4dc, PC 0x80109544 dump_stack+0x0/0x1c
    <6>250: addiu sp,sp,-152
    <6>346: frame_size 152
    <6> SP 0x88171b18, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
    <6>250: addiu sp,sp,-24
    <6>346: frame_size 24
    <6> SP 0x88171b30, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
    <6>250: addiu sp,sp,-24
    <6>346: frame_size 24
    <6> SP 0x88171b48, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
    <6>250: addiu sp,sp,-24
    <6>346: frame_size 24
    <6> SP 0x88171b60, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c
    <6>250: addiu sp,sp,-24
    <6>346: frame_size 24
    <6> SP 0x88171b78, RA 0x8010b4dc, PC 0x8010b4dc smp_call_function+0x1c0/0x20c


  • 8.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-22-2017 15:01
    This is just stack fuction call. Is this panic hapenning constantly - or you just have one time panic occurance? You can look at the timestamp and see how long ago panic occured. Again - you said MUs are not connecting and unless your controller is constantly rebooting - that panic is not what's causing you a problem. Probably something else in the config.


  • 9.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 12:10
    Hi again Phil,

    Sorry for the confusion. What I meant is, if the controller with MiNT ID 70:38:0A:F0 has the master status on the cluster showing up as true. Can you please share the output of the following command?

    #show cluster members



  • 10.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 12:10
    rfs7000-Backup 70.38.0A.F9 00-15-70-38-0A-F9 False standby 00:01:01 ago
    rfs7000-Primary 70.81.BE.8E 00-15-70-81-BE-8E True active self



  • 11.  RE: RFS7000 Crashing - Wing 5.8.5

    Posted 09-20-2017 12:10
    rfs7000-Backup 70.38.0A.F9 00-15-70-38-0A-F9 False standby 00:01:01 ago
    rfs7000-Primary 70.81.BE.8E 00-15-70-81-BE-8E True active self