5520 EXOS wont link up to Cisco 3750x 802.11q trunk

[Keith9]

I have a Cisco 3750x 2 switch stack on the second floor of one of our offices and we leased a small suite in the 1st floor of this building for training purposes.  We had the cabling vendor run mm fiber between floor 1 and 2.  For a few months the 1st floor was connected by 2 x 1 gbps fiber connections to the second floor using a Cisco 3650 switch.  Two WAN connections come into the second floor cisco switch stack and use OSPF to get back to HQ.  The EXOS switch on the 1st floor is just an access switch extension off of the 2nd floor.  The config looks like this:

interface Port-channel1
 description Trunk to 1566 2nd Floor
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 spanning-tree bpdufilter enable
 ip dhcp snooping trust
!
interface GigabitEthernet1/1
 description uplink to 1566 Fl2 Switch 1 Gi1/1/4
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 mls qos trust dscp
 spanning-tree portfast edge
 spanning-tree bpdufilter enable
 channel-group 1 mode active
 ip dhcp snooping trust
!
interface GigabitEthernet1/2
 description uplink to 1566 Fl2 Switch 2 Gi1/1/4
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 mls qos trust dscp
 spanning-tree portfast edge
 spanning-tree bpdufilter enable
 channel-group 1 mode active
 ip dhcp snooping trust
!

Now that switch although it works, its quite a number of years old, so we are replacing it with a 5520-24W-EXOS running EXOS 31.1.1.3.  I put the 4 port NIM in the right side of this switch and in the first two ports I populated them with SFP 1 Gbps 850 nm devices.  I attempted to swing the uplinks to these ports with this configuration, and the ports show a 1000mbps link, there is NO traffic flow on it, and the 2nd floor switch shows its port-channel down.

enable diffserv examination port 33
disable dot1p examination port 33
configure ports 33 description-string "Uplink to 1566 2nd Floor"
configure ports 33 auto off speed 1000 duplex full
enable sharing 33 grouping 33-34 algorithm address-based L3_L4 lacp
configure vlan Default add ports 1-33,35-36 untagged
configure vlan VOICE add ports 1-33,35-36 tagged
configure qosprofile QP3 minbw 0 maxbw 100 ports 33
configure qosprofile QP5 minbw 0 maxbw 100 ports 33
disable edp ports 33
enable cdp ports 33
configure vlan default delete ports 34
enable diffserv examination port 34
disable dot1p examination port 34
configure ports 34 description-string "Uplink to 1566 2nd Floor"
configure ports 34 auto off speed 1000 duplex full
enable sharing 33 grouping 33-34 algorithm address-based L3_L4 lacp
disable edp ports 34
enable cdp ports 34
 

Even with just ONE of the fibers connected, though the port lights and the sh port command indicates a link… traffic will not flow between the EXOS and Cisco switches.

I never had this issue with 450G2’s or 690X EXOS switches, but in those cases I am either connecting two or more ports to a server, or in case of our core, the 690X’s I am doing MLAGs from all other switches in the HQ building.  The 5520 is a new switch for us, and we ordered 14 of them… this is our one 24 port (the rest are 48 and have plans to outfit other buildings with them).  All switches were ordered with the 4 port expantion on the side since this model does not have those ports by default like the previous generation.

The 2nd floor of this building will eventually get its two Cisco 3750X stacked switches replaced with 2 5520’s stacked and the optics switched out for 10 gig on both ends.  It’s one gig now because thats all the cisco supports.

Any idea’s what I did incorrectly?  This is fairly simple, vlan 1 is data, vlan 172 is voip.  Its a small installation.

 sh port 33 vid
         Untagged
Port     /Tagged   VID(s)
-------- --------  ------------------------------------------------------------
33       Untagged  1
         Tagged    172
sh port 34 vid
         Untagged
Port     /Tagged   VID(s)
-------- --------  ------------------------------------------------------------
34       Untagged  1
         Tagged    172
sh sharing
Load Sharing Monitor
Config    Current Agg     Min    Ld Share        Ld Share  Agg Link  Link Up
Master    Master  Control Active Algorithm Flags Group     Mbr State Transitions
================================================================================
    33            LACP       1    L3_L4     A     33        -     R       7
                                  L3_L4           34        -     R       7
================================================================================

[Stefan_K_]

I think he won’t need the MLAG part… Just saying this so he isn’t confused. 🙂 

[Miguel-Angel_RO]

Keith,

Here a working config I had between Cisco and Exos:

Cisco:

interface Ethernet1/17

  description switch-Extreme

  switchport mode trunk

  switchport trunk native vlan 2

  switchport trunk allowed vlan 2-1000

  spanning-tree link-type point-to-point

  flowcontrol receive on

  flowcontrol send on

  channel-group 19 mode active

  no vtp

Exos (cluster):

enable sharing 16 grouping 16 algorithm address-based L2 lacp

create mlag peer "CORE_L3"

configure mlag peer "CORE_L3" ipaddress x.y.z.w vr VR-Default

enable mlag port 16 peer "CORE_L3" id 16

To be checked with your config

Mig

[Keith9]

I think this is 0 because i disabled the sharing and re-enabled it.  I’m not on-site to swing the cables and if I were I would have to wait until after 4:30 PM EST.

I think by default the cisco is 

sh lacp counters

LACP PDUs dropped on non-LACP ports : 0
LACP Bulk checkpointed msgs sent    : 0
LACP Bulk checkpointed msgs recv    : 0
LACP PDUs checkpointed sent         : 0
LACP PDUs checkpointed recv         : 0

Lag        Member     Rx       Rx Drop  Rx Drop  Rx Drop  Tx       Tx
Group      Port       Ok       PDU Err  Not Up   Same MAC Sent Ok  Xmit Err
--------------------------------------------------------------------------------
33         33         0        0        0        0        0        0
           34         0        0        0        0        0        0

================================================================================

On the Cisco side right now here is the port-channel detail.  You’ll notice the last time the port-channel bundled or un-bundled was last Friday late afternoon when I attempted to swing the cables from Cisco to EXOS.  

sh etherchannel 2 detail
Group state = L2
Ports: 2   Maxports = 16
Port-channels: 1 Max Port-channels = 16
Protocol:   LACP
Minimum Links: 0

                Ports in the group:
                -------------------
Port: Gi1/1/4
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 2           Mode = Active          Gcchange = -
Port-channel  = Po2         GC   =   -             Pseudo port-channel = Po2
Port index    = 0           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi1/1/4   SA      bndl      32768         0x2       0x2     0x135       0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi1/1/4   SA      32768     0007.7db6.e580   5s    0x0    0x1    0x11A   0x3D

Age of the port in the current state: 2d:16h:22m:28s

Port: Gi2/1/4
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 2           Mode = Active          Gcchange = -
Port-channel  = Po2         GC   =   -             Pseudo port-channel = Po2
Port index    = 0           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi2/1/4   SA      bndl      32768         0x2       0x2     0x235       0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi2/1/4   SA      32768     0007.7db6.e580   1s    0x0    0x1    0x11B   0x3D

Age of the port in the current state: 2d:16h:22m:21s

                Port-channels in the group:
                ---------------------------

Port-channel: Po2    (Primary Aggregator)

------------

Age of the Port-channel   = 102d:23h:36m:47s
Logical slot/port   = 10/2          Number of ports = 2
HotStandBy port = null
Port state          = Port-channel Ag-Inuse
Protocol            =   LACP
Port security       = Disabled
Load share deferral = Disabled

Ports in the Port-channel:

Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     00     Gi1/1/4  Active             0
  0     00     Gi2/1/4  Active             0

Time since last port bundled:    2d:16h:22m:21s    Gi2/1/4
Time since last port Un-bundled: 2d:16h:27m:53s    Gi1/1/4
 

[Stefan_K_]

This is normal, since Port 33 is the load-sharing master and the VLAN config on this port applies on the whole LAG. 

Well, it should match the configuration on the Cisco Switch. Talking about the cisco switch, is LACP activated there, since you are using LACP on the Extreme-Switch? 
I don’t know cisco very well, but I think channel-group mode needs to be auto in order to use lacp and you need to specify a load-balance algorithm. (e.g. src-dst-ip).

I’m still missing show lacp counters?

[Keith9]

EXOS STP configuration.  since port 33 is in a sharing group I suppose that's why disable stpd s0 ports 33 does not also indicate port 34.  Yes in fact that must be true because I tried to apply it and it says port 34 is not a member of STP domain s0, which makes sense because of treating that as one port with the sharing command.

sh configuration | i stpd
configure stpd s0 priority 0
enable stpd s0 auto-bind vlan VOICE
disable stpd s0 ports 33

sh stp
                MSTP Global Configuration:
MSTP Region Name        : 40882fe15400
MSTP Format Identifier  : 0
MSTP Revision Level     : 3
MSTP Digest             : ac:36:17:7f:50:28:3c:d4:b8:38:21:d8:ab:26:de:62
Common and Internal Spanning Tree (CIST)        : s0
Total Number of MST Instances (MSTI)            : 0

Name       Tag  Flags  Ports Bridge ID        Designated Root  Rt Port Rt Cost
s0         0000 EM----    35 000040882fe15400 000040882fe15400 -------       0

Total number of STPDs: 1                STP Flush Method: VLAN and Port
STP Filter Method: System-wide          STP BPDU Forwarding: On
STP Multicast Send IGMP or MLD Query: On

Flags: (C) Topology Change, (D) Disable, (E) Enable, (R) Rapid Root Failover
       (T) Topology Change Detected, (M) MSTP CIST, (I) MSTP MSTI
 

The only other concern I have is if that 4 port NIM in the side can accept 1GBPS SFP’s.  It links, we get lights and we get the confirmation on the sh port 33-34 of a layer 2 link… just no traffic.  They will be swapped out for 10 gig SFP+ optics once the switch on the 2nd floor is converted over.  The 2nd floor switch is more complicated because its 96 ports, mostly used, trunks to a vmware, two wan links, OSPF, etc.  It will take more downtime and planning, but it is on the 2021 project adjenda.