5520 EXOS wont link up to Cisco 3750x 802.11q trunk

[Keith9]

I have a Cisco 3750x 2 switch stack on the second floor of one of our offices and we leased a small suite in the 1st floor of this building for training purposes.  We had the cabling vendor run mm fiber between floor 1 and 2.  For a few months the 1st floor was connected by 2 x 1 gbps fiber connections to the second floor using a Cisco 3650 switch.  Two WAN connections come into the second floor cisco switch stack and use OSPF to get back to HQ.  The EXOS switch on the 1st floor is just an access switch extension off of the 2nd floor.  The config looks like this:

interface Port-channel1
 description Trunk to 1566 2nd Floor
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 spanning-tree bpdufilter enable
 ip dhcp snooping trust
!
interface GigabitEthernet1/1
 description uplink to 1566 Fl2 Switch 1 Gi1/1/4
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 mls qos trust dscp
 spanning-tree portfast edge
 spanning-tree bpdufilter enable
 channel-group 1 mode active
 ip dhcp snooping trust
!
interface GigabitEthernet1/2
 description uplink to 1566 Fl2 Switch 2 Gi1/1/4
 switchport trunk allowed vlan 1,172
 switchport trunk encapsulation dot1q
 switchport mode trunk
 ip arp inspection trust
 mls qos trust dscp
 spanning-tree portfast edge
 spanning-tree bpdufilter enable
 channel-group 1 mode active
 ip dhcp snooping trust
!

Now that switch although it works, its quite a number of years old, so we are replacing it with a 5520-24W-EXOS running EXOS 31.1.1.3.  I put the 4 port NIM in the right side of this switch and in the first two ports I populated them with SFP 1 Gbps 850 nm devices.  I attempted to swing the uplinks to these ports with this configuration, and the ports show a 1000mbps link, there is NO traffic flow on it, and the 2nd floor switch shows its port-channel down.

enable diffserv examination port 33
disable dot1p examination port 33
configure ports 33 description-string "Uplink to 1566 2nd Floor"
configure ports 33 auto off speed 1000 duplex full
enable sharing 33 grouping 33-34 algorithm address-based L3_L4 lacp
configure vlan Default add ports 1-33,35-36 untagged
configure vlan VOICE add ports 1-33,35-36 tagged
configure qosprofile QP3 minbw 0 maxbw 100 ports 33
configure qosprofile QP5 minbw 0 maxbw 100 ports 33
disable edp ports 33
enable cdp ports 33
configure vlan default delete ports 34
enable diffserv examination port 34
disable dot1p examination port 34
configure ports 34 description-string "Uplink to 1566 2nd Floor"
configure ports 34 auto off speed 1000 duplex full
enable sharing 33 grouping 33-34 algorithm address-based L3_L4 lacp
disable edp ports 34
enable cdp ports 34
 

Even with just ONE of the fibers connected, though the port lights and the sh port command indicates a link… traffic will not flow between the EXOS and Cisco switches.

I never had this issue with 450G2’s or 690X EXOS switches, but in those cases I am either connecting two or more ports to a server, or in case of our core, the 690X’s I am doing MLAGs from all other switches in the HQ building.  The 5520 is a new switch for us, and we ordered 14 of them… this is our one 24 port (the rest are 48 and have plans to outfit other buildings with them).  All switches were ordered with the 4 port expantion on the side since this model does not have those ports by default like the previous generation.

The 2nd floor of this building will eventually get its two Cisco 3750X stacked switches replaced with 2 5520’s stacked and the optics switched out for 10 gig on both ends.  It’s one gig now because thats all the cisco supports.

Any idea’s what I did incorrectly?  This is fairly simple, vlan 1 is data, vlan 172 is voip.  Its a small installation.

 sh port 33 vid
         Untagged
Port     /Tagged   VID(s)
-------- --------  ------------------------------------------------------------
33       Untagged  1
         Tagged    172
sh port 34 vid
         Untagged
Port     /Tagged   VID(s)
-------- --------  ------------------------------------------------------------
34       Untagged  1
         Tagged    172
sh sharing
Load Sharing Monitor
Config    Current Agg     Min    Ld Share        Ld Share  Agg Link  Link Up
Master    Master  Control Active Algorithm Flags Group     Mbr State Transitions
================================================================================
    33            LACP       1    L3_L4     A     33        -     R       7
                                  L3_L4           34        -     R       7
================================================================================

[Keith9]

If you use standalone ESXi you have to create static LAG on EXOS. ESXi do not support LACP. In order to have LACP based LAG you should have VMware Distributed Switch. Available with VWmare Enterprise Plus license or with NSX.

eg: 

enable sharing 1 grouping 1,2 algorithm address-based L2

Mind no “lacp” statement at the end.

Ah yes, in our HQ we have vcenter and therefore distributed switch capability, so that is setup.  But at a branch office its just 1 1U server running free ESXi, so no need for vcenter when its just 1 system.  But yeah its more about redundancy than load balancing.  The small number of users at a branch office never need tons of bandwidth, just reliability.

[Adam_Minowski]

If you use standalone ESXi you have to create static LAG on EXOS. ESXi do not support LACP. In order to have LACP based LAG you should have VMware Distributed Switch. Available with VWmare Enterprise Plus license or with NSX.

eg: 

enable sharing 1 grouping 1,2 algorithm address-based L2

Mind no “lacp” statement at the end.

[Adam_Minowski]

In most cases HA is more important than LB. Of course proper balancing is important to have most of a “value” from multiple links, but LACP algorithm is irrelevant when LACP is not working at all 😉

About issue itself. Also I’d check LACP passive mode on Cisco switch, and first of all log files on both sides, plus LACP debug.

From Cisco documentation:

“Both the active and passive LACP modes enable ports to negotiate with partner ports to an EtherChannel based on criteria such as port speed, and for Layer 2 EtherChannels, based on trunk state and VLAN numbers. Ports can form an EtherChannel when they are in different LACP modes as long as the modes are compatible”.

As a closing note. If I understand correctly, similar configuation worked witch Cisco on older EXOS switches - in that case I’d suggest to contact GTAC.

[Keith9]

I’m not convinced that 1 gig sfp’s work properly.  The same config I tried at this site worked fine in another site, from copper ports to copper ports, and 10 gig sfp+ ports to 10 gig sfp+ (populated with 850nm mm fiber) to a server.

I’ll be out of the office until January 4th and sometime in Q1 we will redo the switching in the second floor there and it will be from Extreme to Extreme, 10 gig SFP+ fiber optic modules.