HELP! Stack unexpected reboot

[Mykhaylo_Skrypk]

XOS: ExtremeXOS version 15.3.3.5 v1533b5-patch1-2

One of our extreme x460/x450 stacks rebooted unexpectedly this morning (at 04:52 ) Logs suggests the following:

2017-01-18 04:57:14.33 Stacking port 1:1 link up at 10Gbps.
2017-01-18 04:57:13.99 Starting hal initialization ....
2017-01-18 04:57:12.29 telnetd listening on port 23

2017-01-18 04:57:06.18 The stack MAC address is not correctly configured on this node. The stack can not operate properly in this condition. Please correct and reboot.
2017-01-18 04:57:03.16 DM started
2017-01-18 04:57:02.95 The Node Manager (NM) has started processing.
2017-01-18 04:57:02.15 EPM Started
2017-01-18 04:57:01.83 Changing to watchdog warm reset mode
2017-01-18 04:52:20.87 Slot-1 FAILED (1) Backup lost
2017-01-18 04:52:20.83 Shutting down all processes
2017-01-18 04:52:20.53 Node State[4] = FAIL (Backup lost)
2017-01-18 04:52:20.53 MASTER decided that I am not BACKUP anymore
2017-01-18 04:52:20.53 BACKUP NODE (Slot-1) DOWN

has anyone had a similar problem?
Thx,
Mykhaylo

[Mykhaylo_Skrypk]

Yeah seen these 12 CRC errors and will do as advised. Thanks. The logs are actually already from the slot 1. Because when i am trying to telnet to slot 1 getting expected error: Error: Cannot establish connection to self.

[Vellachery__Sum]

Mykhaylo,

Adding to it, did you get an opportunity to review the logs from Slot-1 at the time of issue occurrence?

Also, ensure that we rule out physical layer related issues (isolating the power cord/power source).

[Vellachery__Sum]

Mykhaylo,

Have an eye on the stack-ports rxerrors. As of now i only see CRC errors on stacking port 1:2 as shown below.

Try clearing the counters and monitor the rxerrors on the stacking ports. In case, if you notice them getting frequently.
incremented. Swap the stacking cables and monitor them once again.

[Mykhaylo_Skrypk]

Would be nice to have an option attaching .txt files while writing a reply as output takes a lot of screen space! Just a thought

[Mykhaylo_Skrypk]

Looks like no errors:

Slot-1 es-vil-vbwc-20.3 # show switch | i nt.TCurrent Time: Wed Jan 18 17:46:43 2017
Slot-1 es-vil-vbwc-20.4 # show stacking detail
Stacking Node 00:04:96:83:4c:a6 information:
Current:
Stacking : Enabled
Role : Master
Priority : Automatic
Slot number : 1
Stack state : Active
Master capable? : Yes
Stacking protocol : Standard
License level restriction :
In active topology? : Yes
Factory MAC address : 00:04:96:83:4c:a6
Stack MAC address : 02:04:96:83:4c:a6
Alternate IP address :
Alternate gateway :
Stack Port 1:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Stack Port 2:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Configured:
Stacking : Enabled
Master capable? : Yes
Slot number : 1
Stack MAC address : 02:04:96:83:4c:a6
Stacking protocol : Standard
License level restriction :
Stack Port 1:
Selection : Native
Stack Port 2:
Selection : Native

Stacking Node 00:04:96:83:4c:a8 information:
Current:
Stacking : Enabled
Role : Backup
Priority : Automatic
Slot number : 2
Stack state : Active
Master capable? : Yes
Stacking protocol : Standard
License level restriction :
In active topology? : Yes
Factory MAC address : 00:04:96:83:4c:a8
Stack MAC address : 02:04:96:83:4c:a6
Alternate IP address :
Alternate gateway :
Stack Port 1:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Stack Port 2:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Configured:
Stacking : Enabled
Master capable? : Yes
Slot number : 2
Stack MAC address : 02:04:96:83:4c:a6
Stacking protocol : Standard
License level restriction :
Stack Port 1:
Selection : Native
Stack Port 2:
Selection : Native

Stacking Node 00:04:96:36:a2:53 information:
Current:
Stacking : Enabled
Role : Standby
Priority : Automatic
Slot number : 3
Stack state : Active
Master capable? : Yes
Stacking protocol : Standard
License level restriction :
In active topology? : Yes
Factory MAC address : 00:04:96:36:a2:53
Stack MAC address : 02:04:96:83:4c:a6
Alternate IP address :
Alternate gateway :
Stack Port 1:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Stack Port 2:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Configured:
Stacking : Enabled
Master capable? : Yes
Slot number : 3
Stack MAC address : 02:04:96:83:4c:a6
Stacking protocol : Standard
License level restriction :
Stack Port 1:
Selection : Native
Stack Port 2:
Selection : Native

Stacking Node 00:04:96:83:4c:2e information:
Current:
Stacking : Enabled
Role : Standby
Priority : Automatic
Slot number : 4
Stack state : Active
Master capable? : No
Stacking protocol : Standard
License level restriction :
In active topology? : Yes
Factory MAC address : 00:04:96:83:4c:2e
Stack MAC address : 02:04:96:83:4c:a6
Alternate IP address :
Alternate gateway :
Stack Port 1:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Stack Port 2:
State : Operational
Blocked? : Yes
Control path active? : Yes
Selection : Native
Configured:
Stacking : Enabled
Master capable? : No
Slot number : 4
Stack MAC address : 02:04:96:83:4c:a6
Stacking protocol : Standard
License level restriction :
Stack Port 1:
Selection : Native
Stack Port 2:
Selection : Native

Stacking Node 00:04:96:35:cf:25 information:
Current:
Stacking : Enabled
Role : Standby
Priority : Automatic
Slot number : 5
Stack state : Active
Master capable? : Yes
Stacking protocol : Standard
License level restriction :
In active topology? : Yes
Factory MAC address : 00:04:96:35:cf:25
Stack MAC address :
Alternate IP address :
Alternate gateway :
Stack Port 1:
State : Operational
Blocked? : Yes
Control path active? : Yes
Selection : Native
Stack Port 2:
State : Operational
Blocked? : No
Control path active? : Yes
Selection : Native
Configured:
Stacking : Enabled
Master capable? : Yes
Slot number : 5
Stack MAC address :
Stacking protocol : Standard
License level restriction :
Stack Port 1:
Selection : Native
Stack Port 2:
Selection : Native
Slot-1 es-vil-vbwc-20.5 #
Slot-1 es-vil-vbwc-20.5 #
Slot-1 es-vil-vbwc-20.5 # show ports stack-ports rxerrors no-refresh
Port Rx Error Monitor
Port Link Rx Rx Rx Rx Rx Rx Rx
State Crc Over Under Frag Jabber Align Lost
================================================================================
1:1 A 0 0 0 0 0 0 0
1:2 A 12 0 0 0 0 0 0
2:1 A 0 0 0 0 0 0 0
2:2 A 0 0 0 0 0 0 0
3:1 A 0 0 0 0 0 0 0
3:2 A 0 0 0 0 0 0 0
4:1 A 0 0 0 0 0 0 0
4:2 A 0 0 0 0 0 0 0
5:1 A 0 0 0 0 0 0 0
5:2 A 0 0 0 0 0 0 0
================================================================================
Link State: A-Active, R-Ready, NP-Port Not Present L-Loopback
Slot-1 es-vil-vbwc-20.6 # show ports stack-ports txerrors no-refresh
Port Tx Error Monitor
Port Link Tx Tx Tx Tx Tx Tx
State Coll Late coll Deferred Errors Lost Parity
================================================================================
1:1 A 0 0 0 0 0 0
1:2 A 0 0 0 0 0 0
2:1 A 0 0 0 0 0 0
2:2 A 0 0 0 0 0 0
3:1 A 0 0 0 0 0 0
3:2 A 0 0 0 0 0 0
4:1 A 0 0 0 0 0 0
4:2 A 0 0 0 0 0 0
5:1 A 0 0 0 0 0 0
5:2 A 0 0 0 0 0 0
================================================================================
> indicates Port Display Name truncated past 8 characters
Link State: A-Active, R-Ready, NP-Port Not Present L-Loopback
Slot-1 es-vil-vbwc-20.7 #
Slot-1 es-vil-vbwc-20.7 # show switch | i nt.T
Current Time: Wed Jan 18 17:47:10 2017
Slot-1 es-vil-vbwc-20.8 #