Odd output of sh ipstats

  • 0
  • 1
  • Question
  • Updated 2 years ago
  • Answered
  • (Edited)
System Type:      X460-48t

Switch      : 800538-00-05 1450N-xxxxx Rev 5.0 BootROM: 2.0.2.3    IMG: 15.5.2.9
PSU-1       : EDPS-300AB A-S9 800515-00-03 1450E-xxxxx
PSU-2       : EDPS-300AB A-00 800515-00-05 1549E-xxxxx

Image   : ExtremeXOS version 15.5.2.9 v1552b9 by release-manager on Wed Jul 23 00:48:16 EDT 2014
BootROM : 2.0.2.3  Diagnostics : 6.3

Trying to solve a different issue I stumbled upon the output of sh ipstats, every single int had errors and discards, at almost the same amount.  We are seeing no physical rx or tx issue on sw, so what would cause every single int to have errors/discards? Sw rebooted 7 days ago and I cleared counters last night.

IP Global Statistics
InReceives =     408019 InUnicast  =     101240 InBcast    =     189297
                        InMcast    =     117482
InHdrErr   =          2 Bad vers   =          0 Bad chksum =          2
                        Short pkt  =          0 Short hdr  =          0
                        Bad hdrlen =          0 Bad length =          0
InDelivers =     293678 InDiscards =          0 Bad Proto  =          0
OutRequest =     226817 OutDiscard =          0 OutNoRoute =          0
Forwards   =      36516 ForwardOK  =      36516 Fwd Err    =          0
NoFwding   =          0 Redirects  =          0 No route   =          0
Bad TTL    =          0 Bad MC TTL =      12448
Bad IPdest =          0 Blackhole  =          0 Output err =          0
MartianSrc =          0
FragCreate =        912 FragOKs    =        233 FragFails  =          0

Global ICMP Statistics
OutResp    =      29045 OutError   =          0 InBadcode  =          0
InTooshort =          0 Bad chksum =          0 In Badlen  =          0
echo reply                      In =          0        Out =      29045
destination unreachable         In =          7        Out =      35714
 - protocol unreachable         In =          0        Out =          0
redirect                        In =          0        Out =          0
echo                            In =      29045        Out =          0
router advertisement            In =          0        Out =          0
time exceeded                   In =          0        Out =          0

Global IGMP Statistics
Out Query  =       5036 Out Report =        681 Out Leave  =          0
In Query   =       2556 In Report  =      31350 In Leave   =          3
In Error   =          0

Router Interface ClassSvc
 
     Stats:  IN         OUT
              0        1176 packets
              0        1176 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0       37632 errors
              0       37632 discards
              0             unknown protos

Router Interface CoreSvc
     i=
     Stats:  IN         OUT
              0        1568 packets
              0        4014 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0       50176 errors
              0      242562 discards
              0             unknown protos

Router Interface FacilitySvc
'
     Stats:  IN         OUT
              0        1176 packets
              0        1677 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0       37632 errors
              0       63016 discards
              0             unknown protos

Router Interface Loopback0

     Stats:  IN         OUT
              0        1568 packets
              0        4013 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0       50176 errors
              0      242516 discards
              0             unknown protos

Router Interface Manage
  
              0        7388 packets
              0        8327 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0      497202 errors
              0      600801 discards
              0             unknown protos

Router Interface SsscSvc
  
     Stats:  IN         OUT
              0        2953 packets
              0        1911 octets
              0           0 Mcast pkts
              0           0 Bcast pkts
              0      199557 errors
              0       99072 discards
              0             unknown protos
Photo of Jimmy Sands

Jimmy Sands

  • 432 Points 250 badge 2x thumb

Posted 2 years ago

  • 0
  • 1
Photo of Tripathy, Priya Ranjan

Tripathy, Priya Ranjan, ESE

  • 2,306 Points 2k badge 2x thumb
Looking at the above log for show ipstats seems Show ipstats missing inbound stats from ip forwarded interface in Exos 15.5.x versions. For a change can you please try upgrading this switch to the recommended release of EXOS16.1.3.6 latest patch. For more information please check if the below article can help you more:

https://gtacknowledge.extremenetworks.com/articles/Solution/Show-ipstats-missing-inbound-stats-in-15...
(Edited)
Photo of Jimmy Sands

Jimmy Sands

  • 432 Points 250 badge 2x thumb
WOW, thank you...I had not noticed that.  I was concerned the SNMP issue in 'X460 switch upgrade causing CPU spike for SNMP process'.  So this should be fun!! Going to read the release notes and post any questions. ty
Photo of Jimmy Sands

Jimmy Sands

  • 432 Points 250 badge 2x thumb
Can I assume the only file I need to put on the tftp server is :

summitX-16.1.3.6-patch1-11.xos

The download section list 10 files.
(Edited)
Photo of Drew C.

Drew C., Community Manager

  • 38,610 Points 20k badge 2x thumb
Hi Jimmy,
Yes - that's the only file you'll need unless you want to load the SSH module.  The link to request the SSH files is in the same place in the eSupport site.

You may want to go with summitX-16.1.4.2-patch1-7.xos though. It's a more recent roll-up patch than the one mentioned above. It will also include the fix for the ipstats bug.
Photo of Jimmy Sands

Jimmy Sands

  • 432 Points 250 badge 2x thumb

Goooood catch, doing so now.  ty


Photo of Tripathy, Priya Ranjan

Tripathy, Priya Ranjan, ESE

  • 2,306 Points 2k badge 2x thumb
Moreover we believe you have run the below command to execute them:

download image <TFTP server IP> summitX-16.1.4.2-patch1-7.xos vr vr-mgmt
Photo of Jimmy Sands

Jimmy Sands

  • 432 Points 250 badge 2x thumb
Thanks all. We were having the oddest problem at a site with about guesstimating about 60 extreme switches, randomly devices would reboot due to watchdog timer (and task tNetTask was pegging on all of them). 

01/20/2017 07:19:54.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x80a8508c)
01/20/2017 07:09:55.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x807cb294)
01/20/2017 06:59:41.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x807cb27c)
01/20/2017 02:58:56.45 <Crit:SYST> task tNetTask cpu utilization is 83% PC: 80c317d4
01/19/2017 14:29:41.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x807cb294)
01/19/2017 13:59:49.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x807cb340)
01/19/2017 13:29:42.98 <Crit:KERN> [1] System rebooted because watchdog timer expired! (0x807cb294)


But it only happened during times when users were on site.Weekends, nothing, holidays, nothing, weekdays between 7 AM and 2 PM, reboots. That was ultimately the real problem here. After 2.5 days of looking at sniffs from the network and tracing 593467 dead ends, I started looking elsewhere.  Hence sh ipstats question above.  Well, we updated the code Monday evening on the x460 as suggested (and also rebooted every Extreme device on site).  Since then, no reboots and no CPU warnings about tNetTask.  Unfortunately I incorporated two possible fixes in the same action (upgrade and reboots) so I do not know what fixed it.

Its one of those darnest things, why only when students/staff were present and why didnt we detect anything from a sniffer (not just me but two other well verse colleagues) on access and trunk ports on access layer and distro switches).

Regardless, they have had this problem for the past 3 weeks, and it stopped starting Tuesday morning, so thank you!! This was one of the most baffling cases I have seen.
Photo of Tripathy, Priya Ranjan

Tripathy, Priya Ranjan, ESE

  • 2,306 Points 2k badge 2x thumb
Nice to know this has been fixed since last monday code upgrade. Actually for every upgrade we have to reboot the device which would then allow these changes to make happen. So obviously as suggested earlier in the post the code upgrade fixed the issue.
Plz let us know if we can close this hub thread as the issue is fixed already.