topic RE: LACP issues in ExtremeSwitching (EXOS/Switch Engine)

LACP issues

Jason_Hilt — Tue, 31 Jan 2017 22:59:00 GMT

I have some 450G2 switches that I can not get LACP LAG to work on. Core 670 switch is grouping ports 19 and 20. Sh lacp lag displays:
Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
19 0 Current Selected Collect-Dist A-GSCD-- 1049
20 0 Current Selected Attached A-GS---- 2049
================================================================================
Actor Flags: A-Activity, T-Timeout, G-Aggregation, S-Synchronization
C-Collecting, D-Distributing, F-Defaulted, E-Expired

Remote switch shows:
Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
1:49 0 Current Selected Collect-Dist A-GSCD-- 1019
2:49 0 Defaulted Unselected Detached A-G---F- 0
================================================================================
Actor Flags: A-Activity, T-Timeout, G-Aggregation, S-Synchronization
C-Collecting, D-Distributing, F-Defaulted, E-Expired

I have tried moving the fiber cable to different pairs. removed and added the grouping. Disabled ports, renabled ports, all with no affect. Link lights are on both switches.

What can be wrong?

Not sure if this is a connected issue or not, but all of my network switches are also showing a loop detected on the uplink port from the core. ELRP is active on edge switches with the uplink ports excluded.
Slot-1: [CLI:vlan:2] LOOP DETECTED : 846382 transmitted, 9114521 received, ingress slot:port (1:49) egress slot:port (1:49)

Fairly new to Extreme switching so bear with me.

RE: LACP issues

Patrick_Voss — Tue, 31 Jan 2017 23:02:00 GMT

Hello Jason,

What does the "show sharing" display? If LACP wasn't working correctly then it should show the port as active but not added into the aggregator so it will display a "-" instead of a "y". Does it link up when you configure it without LACP? This could be a good test.

RE: LACP issues

Ty_Kolff — Tue, 31 Jan 2017 23:07:00 GMT

What version of EXOS are you using on the 450-G2's?

RE: LACP issues

Jason_Hilt — Tue, 31 Jan 2017 23:17:00 GMT

21.1.1.4 Patch1-2.

sharing shows AGG Mbr -

Can't check without LACP right now while school is in session.

RE: LACP issues

Patrick_Voss — Tue, 31 Jan 2017 23:50:00 GMT

Hello Jason,

So the port is active and has a "y" next to both ports? on both sides?

RE: LACP issues

Jason_Hilt — Wed, 01 Feb 2017 00:22:00 GMT

There is only a Y on one port on each end. The other port shows a - for the Agg Mbr.
Core:
19 19 LACP 1 L2 A 19 Y A 8
L2 20 - A 14
Remote switch:
1:49 1:49 LACP 1 L2 A 1:49 Y A 3
L2 2:49 - A 12
================================================================================

Sh lacp lag on the core is:
Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
19 0 Current Selected Collect-Dist A-GSCD-- 1049
20 0 Current Selected Attached A-GS---- 2049
================================================================================

Sh lacp lag on the remote switch is:
Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
1:49 0 Current Selected Collect-Dist A-GSCD-- 1019
2:49 0 Defaulted Unselected Detached A-G---F- 0
================================================================================

Connections are 10gig fiber on SM fiber using Extreme Network branded SumitomoElectric SFP+ optics.

RE: LACP issues

Patrick_Voss — Wed, 01 Feb 2017 01:05:00 GMT

Hello Jason,

The "A" indicates the port is active. When there is a "Y" it means that it is active in the share group. So in this case port 20 and 2:49 are not passing traffic. The ELRP messages may be unrelated to the LACP issue. I would look into the other side of 1:49 and see if ELRP is being triggered on that switch.

RE: LACP issues

Jason_Hilt — Wed, 01 Feb 2017 01:23:00 GMT

ELRP is not configured on the core switch. Only on edge switches.

RE: LACP issues

Ty_Kolff — Wed, 01 Feb 2017 02:14:00 GMT

We discovered an issue with LACP last week with EXOS version 21.1.2. It's a known issue that will be resolved in 21.1.3, but it hasn't been released yet. In this case, we had to roll back to 16.1.4.2 patch 1-7.

RE: LACP issues

Jason_Hilt — Wed, 01 Feb 2017 02:51:00 GMT

Is this a random issue as we have quite a few other buildings with the same switches, SFPs, and cables that are working just fine running 21.1.1.4?

RE: LACP issues

Ty_Kolff — Wed, 01 Feb 2017 03:03:00 GMT

I have others running just fine on 21.1.1 as well as 21.1.2, but we discovered this issue while doing bench testing before putting these switches into production.

RE: LACP issues

Erik_Auerswald — Thu, 02 Feb 2017 19:09:00 GMT

Hi Jason,

did you verify the cabling with e.g. "show edp port 19,20" resp. "show edp port 1:49,2:49"?

You can use the command "show lacp member-port detail" for more info regarding LACP on the ports.

Erik

RE: LACP issues

Jason_Hilt — Thu, 02 Feb 2017 20:14:00 GMT

sh edp port 1:49,2:49 shows correct remote ports on the core.
Port Neighbor Neighbor-ID Remote Age Num
Port Vlans
=============================================================================
1:49 FC-Core-670 00:00:00:04:96:9b:37:50 1:12 9 15
2:49 FC-Core-670 00:00:00:04:96:9b:37:50 1:11 5 15
=============================================================================

sh lacp lag 1:49 after a disable/enable on port 2:49. RX state went to expired before going to Defaulted.

Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
1:49 0 Current Selected Collect-Dist A-GSCD-- 1012
2:49 0 Defaulted Unselected Detached A-G---F- 0
================================================================================

Going to try reseating the optics...again and maybe a different fiber pair.

RE: LACP issues

Erik_Auerswald — Thu, 02 Feb 2017 20:31:00 GMT

Hi Jason,

with "show lacp member-port 1:49 detail" you get counters regarding sent and received LACP PDUs, and the LACP system IDs.

I would expect that port 2:49 does not receive LACP PDUs.

Erik

RE: LACP issues

Jason_Hilt — Thu, 02 Feb 2017 20:57:00 GMT

So here is the trouble port after a SFP reseat, fiber pair change, and disable/enable.

Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
2:49 0 Defaulted Unselected Detached A-G---F- 0
Up : Yes
Enabled : Yes
Link State : Up
Actor Churn : True
Partner Churn : True
Ready_N : No
Wait pending : No
Ack pending : No
LAG Id:
S.pri:0 , S.id:00:00:00:00:00:00, K:0x0000, P.pri:0 , P.num:0
T.pri:0 , T.id:00:00:00:00:00:00, L:0x0000, Q.pri:0 , Q.num:0
Stats:
Rx - Accepted : 0
Rx - Dropped due to error in verifying PDU : 0
Rx - Dropped due to LACP not being up on this port : 0
Rx - Dropped due to matching own MAC : 0

Tx - Sent successfully : 128
Tx - Transmit error : 0
================================================================================
Actor Flags: A-Activity, T-Timeout, G-Aggregation, S-Synchronization
C-Collecting, D-Distributing, F-Defaulted, E-Expired

Here is the core side of the link

Member Port Rx Sel Mux Actor Partner
Port Priority State Logic State Flags Port
--------------------------------------------------------------------------------
11 0 Current Selected Attached A-GS---- 2049
Up : Yes
Enabled : Yes
Link State : Up
Actor Churn : False
Partner Churn : True
Ready_N : Yes
Wait pending : No
Ack pending : No
LAG Id:
S.pri:0 , S.id:00:04:96:9b:37:50, K:0x03f3, P.pri:0 , P.num:1011
T.pri:0 , T.id:02:04:96:9b:02:c4, L:0x0419, Q.pri:0 , Q.num:2049
Stats:
Rx - Accepted : 268
Rx - Dropped due to error in verifying PDU : 0
Rx - Dropped due to LACP not being up on this port : 0
Rx - Dropped due to matching own MAC : 0

Tx - Sent successfully : 13
Tx - Transmit error : 0
================================================================================
Actor Flags: A-Activity, T-Timeout, G-Aggregation, S-Synchronization
C-Collecting, D-Distributing, F-Defaulted, E-Expired

Going to try a different SFP next.

RE: LACP issues

Erik_Auerswald — Thu, 02 Feb 2017 21:10:00 GMT

So it seems the LACP PDUs from the core switch do not reach the X450, but the other direction works.

Did you verify that the transmission fiber strand from the X450 port 2:49 actually connects to the correct X670 port's receive fiber strand, e.g. by using EDP on the X670 switch?

Did you check the SFP types on both sides (core and access)?

Do you see errors on the ports (e.g. "show port 2:49 rxerrors no-refresh")?

RE: LACP issues

Jason_Hilt — Thu, 02 Feb 2017 22:55:00 GMT

Ports are correct. Disabled remote port 2:49 and port 11 on the core went offline.
SFP info is the same on both ends.
No txerrors or rxerrors on either port.
Changed out SFP with a working connection on both ends. No luck. Changed out fiber on both ends. No luck. Changed fiber pair again - third time. No luck.

RE: LACP issues

Erik_Auerswald — Thu, 02 Feb 2017 23:10:00 GMT

Hm, in the first post you showed ports 19&20 on the core, but EDP showed ports 11&12 on the core. Do you have configured the correct core ports for port sharing with LACP?

RE: LACP issues

Jason_Hilt — Fri, 03 Feb 2017 00:45:00 GMT

Sorry about that. My initial problem was on a different network closet which fixed itself after I rebooted all of the switches that were having this problem. The latest posts deal with the one remaining closet that still isn't working.

RE: LACP issues

Erik_Auerswald — Fri, 03 Feb 2017 00:53:00 GMT

I would suggest opening a case with GTAC, since the configuration looks OK. LACP PDUs are sent on one end, but not received (or registered / correctly interpreted) on the other.