topic RE: Lag configuration question in ExtremeSwitching (EOS)

Lag configuration question

Stephen_McGuire — Mon, 01 Aug 2016 23:24:00 GMT

I have an S4 (main campus) with a lag on one end and I have three stacked B5's (remote campus) on the other. They are connected via 1 mile long fiber.

Complaint from end user's has been poor internet speeds at the remote campus. They are on a VLAN and come back via the fiber to go out through our internet provider.

I just started this job and am new to the extreme EOS though I just finished the bootcamp and passed the switching and routing exam.

I started to just look at the VLAN performance and noted it was terrible and I'm on main campus side. I'm trying to document the existing switching design etc. So I began to dig into how things are connected L1/L2.

I did notice though that I have two ports on the S4 setup as a LAG to the stacked B5's but on the stacked B5's there are NO LAGs configured. Globally LACP is enabled on both ends but the stack hasn't been configured with a lag.

So first question is, how does this even work? I thought that you had to have both ends configured properly for the LAG to work. I know they can form automatically due to the global setting but I can't find anything on the B5's to indicate that has occurred.

Should I go ahead and setup the lag properly on each end with aadminkey's which one side appears to have already?

Port status shows the ports on the remote end to both be UP, all of the lag's appear as down this is the same as the main campus.

So it looks to me that the lag's are not configured and up properly and that neither end is setup.

Sorry for the long story but wanted to get some feed back on the situation. 🙂

Thoughts and suggestions are appreciated!
-Stephen

RE: Lag configuration question

Daniel_Coughlin — Mon, 01 Aug 2016 23:35:00 GMT

Stephen,

I would suggest configuring the SecureStack side correctly. Then use the show lacp commands to make sure it is the way you want it. This article should guide you:How to Configure Dynamic LAGs with LACP on Securestack or Chassis based switches . If you run into problems let us know.

RE: Lag configuration question

Stephen_McGuire — Mon, 01 Aug 2016 23:35:00 GMT

Thanks, I had most of the CLI already but that's a great link.

RE: Lag configuration question

Jeremy_Gibbs — Tue, 02 Aug 2016 00:55:00 GMT

So, start with on the B5

show lacp lag.0.*

Does it show any ports in a LAG?
Do you have single port LAG turned on (Enabled)?

Also, run show neighbors to confirm the uplink is what you think it is from both ends... if it is, check the ports for errors...

On the B5 show port counters ge.2.4

on the S4

show port counters errors nonzero

Are you seeing any errors on the ports or the supposed lag port?

Also, on the S4, for that particular problem VLAN. run the command

show run int vlan.0.20

Paste the output here.

RE: Lag configuration question

Stephen_McGuire — Tue, 02 Aug 2016 00:55:00 GMT

Hello, I'll do my best to answer the questions:
B5, show lacp lag.0.* show no lags with ports setup.

I don't believe I have single port lag enabled.

When I run show neighbors on the B5 I get the following output that is confusing:
180 OCC(su)->show nei
Port Device ID Port ID Type Network Address
---------------------------------------------------------------------------------
ge.1.46 20b3993daec2 ge.1.48 ciscodp 10.99.162.12
ge.1.46 00-1f-45-fb-7d-48 ge.2.112 ciscodp 10.99.0.1
ge.1.46 00:00:00:00:00:00 Gigabit Ethernet Frontpanel port 73 cdp 10.99.180.1
ge.1.46 20:B3:99:3D:AE:C2 ge.1.48 lldp
ge.1.46 00:1F:45:FB:7D:48 ge.2.112 lldp
ge.1.47 00-1f-45-fb-7d-48 ge.3.112 ciscodp 10.99.0.1
ge.1.47 00:1f:45:fb:7d:48 ge.3.112 cdp 10.99.0.1
ge.1.47 00:1F:45:FB:7D:48 ge.3.112 lldp

It looks as if there are more than one devices connected via ge.1.46?

Port counters shows no errors on the B5 or the S4.

Vlan show run from S4:
configure terminal
!
interface vlan.0.91
description "180-OCC"
ip address 10.128.191.1 255.255.255.0 primary
ip helper-address 10.150.0.30
ip helper-address 10.150.0.70
no ip proxy-arp
no ip redirects
no shutdown
exit

RE: Lag configuration question

Daniel_Coughlin — Tue, 02 Aug 2016 00:55:00 GMT

Yes this should be investigated:
ge.1.46 20:B3:99:3D:AE:C2 ge.1.48 lldp
ge.1.46 00:1F:45:FB:7D:48 ge.2.112 lldp

Which connection should 20:B3:99:3D:AE:C2 be on. It is possible that this is stale information or it is real and someone moved a cable. You need to get to the bottom of this.

RE: Lag configuration question

Stephen_McGuire — Tue, 02 Aug 2016 00:55:00 GMT

20:B3:99:3D:AE:C2 search in Netsight shows it as the Netmgmt port for another switch in that remote campus location. This is quite confusing.

Here's the show nei on that switch:
162-OLD-COLONY-1(su)->show nei Port Device ID Port ID Type Network Address
---------------------------------------------------------------------------------
ge.1.33 04:A1:51:A8:A9:6C g1 lldp 10.128.175.160
ge.1.48 20b3993da98e ge.1.46 ciscodp 10.99.180.5
ge.1.48 00-1f-45-fb-7d-48 ge.2.112 ciscodp 10.99.0.1
ge.1.48 00:00:00:00:00:00 Gigabit Ethernet Frontpanel port 89 cdp 10.99.180.3
ge.1.48 20:B3:99:3D:A9:8E ge.1.46 lldp
ge.1.48 00:1F:45:FB:7D:48 ge.2.112 lldp

RE: Lag configuration question

Stephen_McGuire — Tue, 02 Aug 2016 00:55:00 GMT

The links are fiber links, can there be a hub on that port? I don't think so, will have to head to the site and trace cables.

RE: Lag configuration question

Erik_Auerswald — Tue, 02 Aug 2016 00:55:00 GMT

Do you know what device sends a port name of "Gigabit Ethernet Frontpanel port 89" resp. "Gigabit Ethernet Frontpanel port 73"? Ip address 10.99.180.3 resp. 10.99.180.1? Does "show neighbors -verbose PORTSTRING" on the S4 provide useful information?

RE: Lag configuration question

Stephen_McGuire — Tue, 02 Aug 2016 00:55:00 GMT

On the S4 I believe only ge.2.112 and ge.3.112 are connected to the remote site via fiber. I ran the -verbose command on the S4 and got the following output:

itsd-core1-S4(su)->show neighbors -verbose ge.2.112Port ge.2.112
Neighbor : 20-b3-99-3d-a9-8e
System Name : 180-OLD-COLONY
Description : SecureStack B5
Location : 180 OCC 2nd FL Closet
MTU : 0
Last Update : THU JAN 01 00:00:00 1970
LLDP
Chassis Id : 20-b3-99-3d-a9-8e
Port : ge.1.46
Support :
Enabled :
CiscoDP
Device Id : 20b3993da98e
Address : 10.99.180.5
Port : ge.1.46
Version : 2
Primary Management : 10.99.180.5
Duplex : Full Duplex
Power : 0 milliwatts
Support : 0x0b01
Neighbor : 20-b3-99-3d-ae-c2
System Name : 162-OLD-COLONY-1
Description : SecureStack B5
Location : 162-OLD-COLONY
MTU : 0
Last Update : THU JAN 01 00:00:00 1970
LLDP
Chassis Id : 20-b3-99-3d-ae-c2

Port : ge.1.48
Support :
Enabled :
CiscoDP
Device Id : 20b3993daec2
Address : 10.99.162.12
Port : ge.1.48
Version : 2
Primary Management : 10.99.162.12
Duplex : Full Duplex
Power : 0 milliwatts
Support : 0x0b01
Neighbor : 00-01-f4-61-34-00
System Name : 162-OLD-COLONY-1
Description : Matrix 1H582-51 : 03.07.30
Port : Gigabit Ethernet Frontpanel port 81
Last Update : THU JAN 01 00:00:00 1970
CDP
Neighbor IP : 10.99.180.1
Chassis IP : 10.99.180.1
Chassis MAC : 00-01-f4-61-34-00
Device Type : router
Support : igmp, rip, ospf, dvmrp, ieee8021q, gvrp

itsd-core1-S4(su)->show neighbors -verbose ge.3.112
Port ge.3.112
Neighbor : 20-b3-99-3d-a9-8e
System Name : 180-OLD-COLONY
Description : Enterasys Networks, Inc. B5 -- Model B5G124-48P2 Rev
06.81.04.0001
Location : 180 OCC 2nd FL Closet
Port : ge.1.47
MTU : 0
Last Update : THU JAN 01 00:00:00 1970
LLDP
Chassis Id : 20-b3-99-3d-a9-8e
Port : ge.1.47
Support :
Enabled :
CDP
Neighbor IP : 10.99.180.5
Chassis IP : 10.99.180.5
Chassis MAC : 20-b3-99-3d-a9-8e
Device Type : dot1qSwitch
Support : ieee8021q, gvrp, igmpSnoop
CiscoDP
Device Id : 20b3993da98e
Address : 10.99.180.5
Port : ge.1.47
Version : 2
Primary Management : 10.99.180.5
Duplex : Full Duplex
Power : 0 milliwatts
Support : 0x0b01

RE: Lag configuration question

Erik_Auerswald — Tue, 02 Aug 2016 00:55:00 GMT

OK, so the mystery device is an old Enterasys E1 switch.

I would assume the E1 does not understand CiscoDP or LLDP and thus floods it to the other ports (it's L2 multicast).

The E1 switch seems to be between the B5 stack and the S4, with other switches connected to it as well.

RE: Lag configuration question

Stephen_McGuire — Tue, 02 Aug 2016 00:55:00 GMT

I looked at that output several times and completely missed that. LOL

RE: Lag configuration question

Erik_Auerswald — Tue, 02 Aug 2016 13:28:00 GMT

As to why the configuration with a LAG on one end only can "work": EOS automatically falls back to using individual ports if no LACP PDUs are received or there are too few links to form the LAG (e.g. one link w/o singleportlag). Spanning tree will block redundant links and prevent a loop.

The LAG and individual port configurations can differ, so "work" should not be read as "work correctly".

RE: Lag configuration question

Stephen_McGuire — Sat, 13 Aug 2016 00:23:00 GMT

Ok, so here's what I found when I was able to get to the location:
B5 has port ge.1.47 connected to the the S4 via fiber to ge.3.112
E1 has port ge.3.1 connected to the S4 via fiber to ge.2.112

There is a link between the E1 ge.2.2 and the B5 ge.1.46 which explains the neighbor info.

Currently the spanning tree info shows the following:
SID Port State Role Cost Priority
-----------------------------------------------------------
0 ge.2.2 forwarding designated 1 128

SID Port State Role Cost Priority
--- ---------- ---------------- ----------- -------- --------
0 ge.1.46 Discarding Alternate 20000 128

So if I understand this correctly, there is no loop but the two fiber runs are not setup in a LAG configuration.

So my question is this, should I place both fiber runs on the B5, and lag them? I have proposed that and included adding a 3rd fiber run to make a single 3GB lag between the B5 and S4 then have the E1 off the B5 stack (there's 3 switches).

Thoughts on that? It's been approved but I could change it. Fiber is already existing and the only financial cost is getting single mode gbic's and a patch cable (under $75 from amazon).

Would like to hear anyone's opinion's on that change in configuration, thanks.
-Stephen

RE: Lag configuration question

Erik_Auerswald — Mon, 15 Aug 2016 13:24:00 GMT

Hi Stephen,

if you keep the direct connection between S4 and E1, you can reboot the B5 stack without losing connectivity to the remote campus. As such I would rather recommend to just add a new fiber link to the B5 stack, but keep the E1 connections.

I would recommend to enable CiscoDP and LLDP on the E1, or disable it on the respective S4 and B5 ports connecting to the E1.

According to an E1 manual for FW 3.07.xx the E1 supports CiscoDP, but not LLDP. So you should disable LLDP on the S4/B5 Ports connecting to the E1, and enable CiscoDP on the E1.

Thanks,
Erik

RE: Lag configuration question

Stephen_McGuire — Tue, 16 Aug 2016 23:30:00 GMT

Hello Erik, that's a really good consideration, something that made me think about what value that has and if it would really benefit us. Without getting to deep in the details, I think the value of the 3gb pipe vs. the redundancy of the one E1 switch isn't sufficient enough to warrant keeping that connection. The density of the office/classroom space has decreased to the point where we may be able to move most connections to the B5 stack and start removing some of the E1's. We rely more and more on WiFi and the fe.*.* ports are just not even in use or have patch cords available in the spaces they occupy. But I like the way you think!

Assuming we go with the 3 connections as ONE lag for 3gb, is there any value is splitting them across the stack? 1st in #1, 2nd in #2 and 3rd in #3? I thought it was mentioned as a best practice in my EOS bootcamp but my memory is fuzzy.

Thoughts?

-Stephen

RE: Lag configuration question

Erik_Auerswald — Tue, 16 Aug 2016 23:48:00 GMT

Hi Stephen,

splitting the LAG links across the stack adds redundancy against one stack member failing, e.g. because of a defective PSU. This is similar to splitting LAG members across different modules in a chassis based switch.

Br,
Erik

RE: Lag configuration question

Stephen_McGuire — Tue, 16 Aug 2016 23:51:00 GMT

Excellent, our S4 only has two slots occupied plus we're nearly full so finding ports will be tough. However the B5's won't be an issue to spread them around.

Great stuff thanks!

-Stephen