ExtremeSwitching (EXOS)

Expand all | Collapse all

X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

  • 1.  X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-09-2018 14:05
    Suddenly my X770-32q rebooted and when i checked the monitoring i see that the memory went high and rebooted.

    I have enabled the debug mode log so this make the memory to go high?


  • 2.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-09-2018 14:08




  • 3.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-10-2018 14:45
    Did you see if there was s system dump file created? This will give GTAC some clue as to what may have caused the reboot.. Command is
    show debug system-dump


  • 4.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-12-2018 05:57
    Hi, what is EXOS firmware version?

    Could you also observe the state of "show cpu-monitoring" ?
    We have one software issue on EXOS.

    Please see the following article.
    https://gtacknowledge.extremenetworks.com/articles/Solution/High-HAL-CPU-on-X770-with-EXOS-22-2-1

    Best regards,


  • 5.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-10-2018 14:45
    Hi,

    Please find the output,

    X770 # show debug system-dump
    ===============================================
    Switch system dump information
    ===============================================
    core_dump_info storage: 8/3072 used [empty]
    --------------------------------
    System memory depleted.
    time: Sun Mar 11 02:36:52 2018
    Global Memory Stats:
    MemTotal: 1021524 kB
    MemFree: 4012 kB
    Buffers: 16 kB
    Cached: 6396 kB
    Active(anon): 41416 kB
    Inactive(anon): 41552 kB
    Active(file): 3204 kB
    Inactive(file): 3096 kB
    HighTotal: 0 kB
    HighFree: 0 kB
    LowTotal 1021524 kB
    LowFree: 4012 kB
    Committed_AS: 599644 kB
    Top Processes by RSS:
    cliMaster: 14180 kB
    hal: 9692 kB
    etmon: 6264 kB
    cfgmgr: 2056 kB
    vlan: 2180 kB
    netTools: 2500 kB
    xmld: 2772 kB
    idMgr: 2140 kB
    epm: 1040 kB
    emsServer: 1752 kB
    fdb: 1656 kB
    edp: 1044 kB
    rtmgr: 1656 kB
    mcmgr: 1632 kB
    pim: 1220 kB
    snmpSubagent: 1384 kB
    Top Slab Caches by Memory Consumption:
    size-2048: 166956 kB
    size-256: 293740 kB
    size-192: 204772 kB
    size-128: 36700 kB
    size-524288: 9216 kB
    size-65536: 9792 kB
    size-262144: 8192 kB
    size-16384: 4304 kB
    size-4096: 4524 kB
    jffs2_refblock: 3864 kB
    size-131072: 3200 kB
    size-1024: 3900 kB
    size-64: 3240 kB
    UNIX: 1108 kB
    sock_inode_cache: 1044 kB
    size-32768: 1408 kB
    --------------------------------
    System memory depleted.
    time: Sun Mar 11 02:36:57 2018
    Global Memory Stats:
    MemTotal: 1021524 kB
    MemFree: 4036 kB
    Buffers: 16 kB
    Cached: 6344 kB
    Active(anon): 41428 kB
    Inactive(anon): 43148 kB
    Active(file): 3168 kB
    Inactive(file): 3148 kB
    HighTotal: 0 kB
    HighFree: 0 kB
    LowTotal 1021524 kB
    LowFree: 4036 kB
    Committed_AS: 599644 kB
    Top Processes by RSS:
    cliMaster: 14196 kB
    hal: 9700 kB
    etmon: 6264 kB
    cfgmgr: 2056 kB
    vlan: 2180 kB
    netTools: 2500 kB
    xmld: 2776 kB
    idMgr: 2148 kB
    epm: 1040 kB
    emsServer: 1752 kB
    fdb: 1656 kB
    edp: 1044 kB
    rtmgr: 1664 kB
    mcmgr: 1632 kB
    pim: 1228 kB
    snmpSubagent: 1384 kB
    Top Slab Caches by Memory Consumption:
    size-2048: 166956 kB
    size-256: 293740 kB
    size-192: 204772 kB
    size-128: 36700 kB
    size-524288: 9216 kB
    size-65536: 9792 kB
    size-262144: 8192 kB
    size-16384: 4304 kB
    size-4096: 4336 kB
    jffs2_refblock: 3864 kB
    size-131072: 3200 kB
    size-1024: 3900 kB
    size-64: 3240 kB
    UNIX: 1108 kB
    sock_inode_cache: 1044 kB
    size-32768: 1408 kB
    --------------------------------
    System memory depleted.
    time: Sun Mar 11 02:37:02 2018
    Global Memory Stats:
    MemTotal: 1021524 kB
    MemFree: 4016 kB
    Buffers: 16 kB
    Cached: 6420 kB
    Active(anon): 37128 kB
    Inactive(anon): 47272 kB
    Active(file): 3272 kB
    Inactive(file): 3084 kB
    HighTotal: 0 kB
    HighFree: 0 kB
    LowTotal 1021524 kB
    LowFree: 4016 kB
    Committed_AS: 599644 kB
    Top Processes by RSS:
    cliMaster: 14180 kB
    hal: 9692 kB
    etmon: 6264 kB
    cfgmgr: 2056 kB
    vlan: 2180 kB
    netTools: 2500 kB
    xmld: 2772 kB
    idMgr: 2140 kB
    epm: 1040 kB
    emsServer: 1752 kB
    fdb: 1656 kB
    edp: 1044 kB
    rtmgr: 1656 kB
    mcmgr: 1632 kB
    pim: 1220 kB
    snmpSubagent: 1384 kB
    Top Slab Caches by Memory Consumption:
    size-2048: 166960 kB
    size-256: 293740 kB
    size-192: 204772 kB
    size-128: 36704 kB
    size-524288: 9216 kB
    size-65536: 9792 kB
    size-262144: 8192 kB
    size-16384: 4304 kB
    jffs2_refblock: 3864 kB
    size-131072: 3200 kB
    size-4096: 3872 kB
    size-1024: 3968 kB
    size-64: 3240 kB
    UNIX: 1108 kB
    sock_inode_cache: 1044 kB
    size-32768: 1408 kB



  • 6.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-12-2018 05:57
    X770# show cpu-monitoring

    CPU Utilization Statistics - Monitored every 5 seconds
    -----------------------------------------------------------------------

    Process 5 10 30 1 5 30 1 Max Total
    secs secs secs min mins mins hour User/System
    util util util util util util util util CPU Usage
    (%) (%) (%) (%) (%) (%) (%) (%) (secs)
    -----------------------------------------------------------------------

    System 13.9 15.4 15.6 45.8 22.5 17.1 19.9 85.7 95.99 58725.83
    aaa 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.5 3.90 3.97
    acl 0.0 0.0 0.0 0.1 0.1 0.0 0.0 1.2 38.90 97.89
    bfd 0.1 0.0 0.0 0.1 0.1 0.0 0.0 0.8 38.87 56.57
    bgp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    brm 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8 1.19 0.89
    ces 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8 1.26 0.79
    cfgmgr 0.1 0.8 0.2 2.2 0.9 0.8 0.8 20.3 956.80 441.11
    cli 0.0 0.0 0.0 0.6 0.1 0.0 0.0 29.2 7.11 3.65
    devmgr 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 5.72 5.75
    dirser 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.83 0.93
    dosprotect 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.11 0.09
    dot1ag 0.2 0.9 0.3 0.0 0.2 0.2 0.2 2.9 60.67 14.69
    eaps 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.41 1.61
    edp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 28.38 11.81
    elrp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 1.29 0.97
    elsm 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8 1.97 2.54
    ems 0.0 0.0 0.0 0.0 0.0 0.0 0.0 9.6 1.86 1.42
    epm 0.1 0.0 0.0 0.1 0.1 0.1 0.1 14.2 158.61 54.95
    erps 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 2.51 1.69
    esrp 0.0 0.0 0.0 0.1 0.0 0.0 0.0 1.2 7.84 13.68
    ethoam 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.49 1.35
    etmon 0.1 0.0 0.0 0.1 0.1 0.1 0.1 1.6 39.68 33.99
    exacl 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exdhcpsnoop 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exdos 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exfib 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exfipSnoop 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exosmc 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exosq 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exsflow 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exsnoop 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    exsshd 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.35 0.67
    exvlan 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    fcoe 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 2.07 1.10
    fdb 0.5 0.5 0.4 0.2 0.3 0.3 0.3 6.1 953.44 351.70
    gptp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    hal 5.7 5.0 3.4 3.7 3.8 3.7 3.8 64.2 1514.10 9619.55
    hclag 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 1.07 0.95
    idMgr 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 1.23 0.89
    ipSecurity 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.09 1.26
    ipfix 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 1.26 0.80
    isis 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    lacp 0.0 0.0 0.0 0.0 0.1 0.0 0.0 1.2 3.83 4.13
    liftd 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.13 0.14
    lldp 0.0 0.0 0.0 0.6 0.2 0.0 0.0 2.6 27.43 10.67
    mcmgr 0.0 0.0 0.0 1.7 0.6 0.2 0.2 7.9 95.92 192.78
    mpls 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    mrp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 2.26 1.72
    msdp 0.0 0.0 0.0 0.0 0.1 0.0 0.0 1.0 16.53 16.13
    netLogin 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 2.45 1.36
    netTools 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 2.64 1.50
    nettx 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    nodemgr 0.0 0.0 0.0 0.1 0.1 0.1 0.0 0.2 32.35 51.11
    ntp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 1.38 1.46
    ospf 0.0 0.0 0.0 1.7 0.4 0.2 0.2 2.3 65.31 32.14
    ospfv3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 7.94 7.82
    pim 0.0 0.0 0.0 0.2 0.1 0.1 0.0 1.4 89.13 18.30
    poe 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 1.06 0.78
    polMgr 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.23 0.24
    ptpV2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    pwmib 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.14 0.14
    rip 0.0 0.0 0.0 0.1 0.1 0.0 0.0 0.9 19.49 36.61
    r.png 0.0 0.0 0.0 0.1 0.0 0.1 0.1 0.8 16.95 20.38
    rtmgr 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 7.30 4.22
    snmpMaster 0.6 0.3 1.4 6.7 3.8 3.3 3.4 19.9 5070.68 1062.51
    snmpSubagent 0.5 0.2 1.3 9.4 4.6 4.0 4.1 24.9 6896.65 894.53
    stp 0.0 0.0 0.0 1.2 0.2 0.2 0.1 3.2 205.16 77.19
    synce 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.71 2.07
    techSupport 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.88 0.85
    telnetd 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.10 0.09
    tftpd 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.11 0.15
    thttpd 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 0.12 0.09
    trill 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00
    upm 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.77 0.82
    vlan 0.0 0.0 0.0 1.7 0.9 0.9 1.0 8.3 1531.14 345.07
    vmt 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 1.33 1.16
    vrrp 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.7 1.56 0.84
    vsm 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.41 1.17
    xmlc 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 0.13 0.09
    xmld 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.9 0.55 0.29

    X770# show version
    Switch : 800534-00-04 1442N-41167 Rev 4.0 BootROM: 2.0.2.1 IMG: 15.6.3.1
    PSU-1 : Internal PSU-1 800462-00-06 1452W-80418
    PSU-2 : Internal PSU-2 800462-00-06 1452W-80467

    Image : ExtremeXOS version 15.6.3.1 v1563b1-patch1-5 by release-manager
    on Sun Aug 16 21:34:14 EDT 2015
    BootROM : 2.0.2.1
    Diagnostics : 6.3



  • 7.  RE: X770-32q got rebooted and suddenly when i checked the monitoring i see that the memory went high.

    Posted 03-12-2018 05:57
    In monitoring when i see the CPU looks good.