Switch BD 15.6 port peak utilzation moved to 100% only on 1 port in LACP


1 port in LACP TX utilization is reaching 100%, average traffic on both these ports are 17% average.

MPGWLCTC> A 1000 4.09 100.00 8.05 100.00 --> 1:48
MPGWLCTC> A 1000 8.72 8.72 7.95 8.17 --> 2:48

Config Current Agg Ld Share Ld Share Agg Link Link UpMaster Master Control Algorithm Group Mbr State Transitions
==============================================================================
1:47 1:47 LACP L3_L4 1:45 - R 0
L3_L4 1:47 Y A 0
L3_L4 2:47 Y A 0
1:48 1:48 LACP L2 1:46 - R 0
L2 1:48 Y A 0
L2 2:48 Y A 0
========================================================================

Need to know what can be the cause and solution for the issue.

Customer says the services are not affected due to this, but my suspection is that some traffic is tweaked affecting peak utilization to 100% but only on 1 port.

9 replies

Userlevel 4
Your 1:48 sharing is in L2 mode
That means if the source and destination mac are the same -- then traffic could not be distributed between all LACP ports

Change it to L3_L4
Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX
Userlevel 4
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

1st scenario also depends on the settings at the router's side
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

1st Scenarion - l3-l4 working on router

2nd scenation - L2-L2 working on secondary Switch
Userlevel 2
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

Hi Saurabh,

On LACP especially if it consists only 2 ports, no matter hash algorithm you use on both side, can have possibility of mostly using one port if the hash algorithm result the same hash. This possibility can be higher if the combination of MACs, IPs and/or TCP/UDP ports (regarding to hash algorithm you use) is too small. For example, if the traffic is only about 2 servers connection with same TCP session (SCP transfer, etc.), the traffic will always use the same port. So, you should inspect your traffic first. L3+L4 hash algorithm can take advantage if the traffic is related to multiple IPs and multiple TCP/UDP ports due to the better hash result combination than L2 hash algorithm. For example is on connecting hypervisor server on virtualization where each node has multiple VMs with their own IPs and each VM has combined services related to their own TCP/UDP ports.

Best regards,
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

Hi Mrxlazuardin,

Thanks for sharing, also lets say if we increase 1 more in the LAG will it change anything ?

regards

Saurabh
Userlevel 2
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

Hi Saurabh,

It depends on the combination of MAC, IP and TCP/UDP ports of mostly traffic. You still have to inspect the traffic first. Anyway you can try L3+L4 algorithm too for finding if it can give better hash result.

Best regards,
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

Hello Mrxlazuardin.

I've applied, but it comes in fraction of seconds so its difficult to check.
Lets see if things change after applying L3_L4

Thank you

regard
Saurabh
Userlevel 2
Saurabh Makkar wrote:

Hello Nick

There are 2 scenarios here

1st scenario -
Switch to Router
L3-L4 is working

2nd Scenario
Switch to Switch
L2-L2 <>

in both these scenario's in 1 port only i'm getting 100% Peak TX

Hi Saurabh,

You can use a kind of flow analyzer or at least TCPDump/WireShark for inspecting your traffic. You cannot rely only on counter on such burst.

Best regards,

Reply