topic RE: When you use the 'top' command on a stack, which CPU is it showing you? in ExtremeSwitching (Other)

When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Thu, 09 Mar 2017 21:20:00 GMT

We have a 5 switch stack of x460-48p. Our monitoring software, Solarwinds, is alerting us to high CPU usage very frequently and intermittently on this switch stack. However, when I use the 'top' command, I don't see CPU usage as high as what is being reported.

What CPU information is being shown when I run the 'top' command on the master of the stack?

What are some possibilities for the high CPU being reported?

Thanks,
Stan

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Nikunj_Tonthana — Fri, 10 Mar 2017 00:33:00 GMT

Running the 'top' command just displays the CPU utilization of the node it is being run on. Not the entire stack.
I'm not sure how SolarWinds is getting CPU usage information for the switch or stack.

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Jeremy_Gibbs — Fri, 10 Mar 2017 00:42:00 GMT

My monitoring software monitors the CPU of each member of the stack and computes an overall load, but can alert on one individual switch.

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Fri, 10 Mar 2017 00:42:00 GMT

ours apparently doesn't do that...

Is there anything specific that I need to configure for us to be able to monitor that?

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Fri, 10 Mar 2017 00:44:00 GMT

so here is another question....

I've noticed some ports flapping. Would high CPU cause that, or the other way around? it seems to be a bunch of random ports spread across the entire stack

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Jeremy_Gibbs — Fri, 10 Mar 2017 00:44:00 GMT

You could have a loop. You could run elrp-client oneshot to find one. Excessive traffic hitting the slow-lane (cpu) of switch, instead of being moved to ASIC....

What process is high in top?

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Fri, 10 Mar 2017 00:44:00 GMT

typically just ./snmpMaster, ./snmpSubagent, and ./hal

it fluctuates while i'm watching top

I apologize but I don't know what elrp-client is

Also, no users are complaining of performance issues.

I'm beginning to wonder if Solarwinds is sending a false positive alert. While watching top on the master node, I have not seen CPU go above say 55%, but the issue might be on a different node, I haven't figured that out yet.

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Jeremy_Gibbs — Fri, 10 Mar 2017 00:44:00 GMT

Run the elrp-client on various VLANS and all ports.. It will let you know if you have a loop.

Jeremy-x460.1 # configure elrp-client one-shot "WirelessAP" ports all print-and-log
Starting ELRP Poll . . .
# NO LOOP DETECTED # --- vlan "WirelessAP" elrp statistics ---
3 packets transmitted, 0 received, ingress port (nil)
* Jeremy-x460.2 #

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Fri, 10 Mar 2017 00:44:00 GMT

Is this something I can do in a production environment, or something that needs to be done off-hours?

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Sat, 11 Mar 2017 01:31:00 GMT

No loops were detected using ELRP for any of our VLANs.

Any other suggestions?

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Stanley_Riley_J — Sat, 11 Mar 2017 02:56:00 GMT

I also made sure that all edge ports were running edge-safeguard, which they are. So I can check that off of my list.

I'm running out of ideas as to why high CPU utilization is being reported.

/sigh

RE: When you use the 'top' command on a stack, which CPU is it showing you?

Nikunj_Tonthana — Sat, 11 Mar 2017 02:56:00 GMT

It is possible that SolarWinds is giving you a false positive as you suspected. Since you say that you do not see any high utilization when you use top.