When you use the 'top' command on a stack, which CPU is it showing you?

  • 0
  • 1
  • Question
  • Updated 2 years ago
  • Answered
We have a 5 switch stack of x460-48p.  Our monitoring software, Solarwinds, is alerting us to high CPU usage very frequently and intermittently on this switch stack.  However, when I use the 'top' command, I don't see CPU usage as high as what is being reported.

What CPU information is being shown when I run the 'top' command on the master of the stack?

What are some possibilities for the high CPU being reported?

Thanks,
Stan
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb

Posted 2 years ago

  • 0
  • 1
Photo of Nikunj Tonthanahal

Nikunj Tonthanahal, SW Systems Engineer

  • 184 Points 100 badge 2x thumb
Running the 'top' command just displays the CPU utilization of the node it is being run on. Not the entire stack.
I'm not sure how SolarWinds is getting CPU usage information for the switch or stack.
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
My monitoring software monitors the CPU of each member of the stack and computes an overall load, but can alert on one individual switch.  
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
ours apparently doesn't do that...


Is there anything specific that I need to configure for us to be able to monitor that?
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
so here is another question....

I've noticed some ports flapping.  Would high CPU cause that, or the other way around?  it seems to be a bunch of random ports spread across the entire stack
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
You could have a loop.  You could run elrp-client oneshot to find one.  Excessive traffic hitting the slow-lane (cpu) of switch, instead of being moved to ASIC....

What process is high in top? 
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
typically just ./snmpMaster, ./snmpSubagent, and ./hal

it fluctuates while i'm watching top

I apologize but I don't know what elrp-client is

Also, no users are complaining of performance issues.

I'm beginning to wonder if Solarwinds is sending a false positive alert.  While watching top on the master node, I have not seen CPU go above say 55%, but the issue might be on a different node, I haven't figured that out yet.
Photo of Jeremy

Jeremy, Embassador

  • 9,788 Points 5k badge 2x thumb
Run the elrp-client on various VLANS and all ports.. It will let you know if you have a loop.



    Jeremy-x460.1 # configure elrp-client one-shot "WirelessAP" ports all print-and-log 
    Starting ELRP Poll . . .
    # NO LOOP DETECTED # --- vlan "WirelessAP" elrp statistics ---
    3 packets transmitted, 0 received, ingress port (nil)
    * Jeremy-x460.2 # 
(Edited)
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
Is this something I can do in a production environment, or something that needs to be done off-hours?
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
No loops were detected using ELRP for any of our VLANs.

Any other suggestions?
Photo of Stanley Riley Jr.

Stanley Riley Jr.

  • 950 Points 500 badge 2x thumb
I also made sure that all edge ports were running edge-safeguard, which they are.  So I can check that off of my list.

I'm running out of ideas as to why high CPU utilization is being reported.

/sigh
Photo of Nikunj Tonthanahal

Nikunj Tonthanahal, SW Systems Engineer

  • 184 Points 100 badge 2x thumb
It is possible that SolarWinds is giving you a false positive as you suspected. Since you say that you do not see any high utilization when you use top.