XOS LACP Share Algorithm L2 how can I make load balance?

  • 0
  • 1
  • Question
  • Updated 2 weeks ago
I have X670 Core with LAGS set up to edge using multicast traffic, but traffic is not balancing using L2 or L3. Is there an extra config needed to balance traffic?
Photo of Keith Olmstead

Keith Olmstead

  • 136 Points 100 badge 2x thumb

Posted 2 weeks ago

  • 0
  • 1
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 48,894 Points 20k badge 2x thumb
Please provide a screenshot of "show sharing detail".
Photo of Keith Olmstead

Keith Olmstead

  • 136 Points 100 badge 2x thumb

# sh sharing detail
Load Sharing Monitor
Config    Current Agg     Min    Ld Share  Dist  Ld Share  Agg Link  Link Up
Master    Master  Control Active Algorithm Flags Group     Mbr State Transitions
================================================================================
  1:43   1:43     LACP       1    L3        A     1:43      Y     A       0
                                  L3              1:44      Y     A       0
==============================================================================

Link Utilization Averages                            Tue Sep 11 12:47:21 2018
Port     Link    Link   Rx             Peak Rx       Tx            Peak Tx
         State   Speed  % bandwidth    % bandwidth   % bandwidth   % bandwidth
================================================================================
1:43      A       10000     1.50          4.89         70.07           84.79
1:44      A       10000     1.74          7.47         22.23           27.18

Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 48,894 Points 20k badge 2x thumb
It's easier to see on a screenshot (as copy/paste messes up the format) but as far as I'd see flag for Ld Share Algorithm is L3 = Layer 3 address based.

You'd change it with this command....

Switch_1.2 # configure sharing <x> algorithm address-based ?
  L2              Based on layer 2 MAC (default)
  L3              Based on layer 3 IP
  L3_L4           Based on layer 3 IP and layer 4 port
  custom          Based on selections in 'configure sharing address-based custom'


Photo of Keith Olmstead

Keith Olmstead

  • 136 Points 100 badge 2x thumb
Ya this copy and paste was after I changed it from L2 to L3 to see if there was any difference in traffic, and there was not.
Photo of EtherMAN

EtherMAN, Embassador

  • 6,960 Points 5k badge 2x thumb
You can use L3_L4 for deepest inspection of packets to try and get a better share.  From my experience you don't get a great share from IPTV type streams between two routers as the macs and such for each are the same on both sides.  Port is not a good way to do traffic balancing.  This has been the biggest reason for use in our ISP metro network to move from 4x10 lag ports to 100 gig interfaces and 870's ... Even the l3_l4 lags we had big internet users connecting their core router to ours and could generate flows at 5 or 6 gigs pushing a single ten gig interface in a lag group into the 90% usage range and the other 3 would be at 10% ... Just remember the more individual macs and flows you have the better the traffic will be spread across the lag group...
Photo of Keith Olmstead

Keith Olmstead

  • 116 Points 100 badge 2x thumb
Thanks for the reply. Pardon my ignorance but what do mean by ports not a good way to balance traffic. Also when you mention flows, are you referring to the physical connections or is that a programming function.
Photo of EtherMAN

EtherMAN, Embassador

  • 6,960 Points 5k badge 2x thumb
Shared ports is what I meant to say.  Flow are the layer 2/3/4 traffic going through the ports from ingress to egress on your shared ports.  From my understanding of how the the sharing algorithm works is ...

Layer 2 it looks at source and destination mac for each of your tcp or udp connections and shares this across the port shared group... So if there is a 2 port share and you have 2 sets of source and destination macs one set would be on one port and the next set would be on the other port.  Layer 3 also looks not only at macs but at ip source and destination so you have one more way to split your traffic apart.  Layer 3-4 also looks at layer 4 info to break apart traffic. 

So you can see if you were using only layer 2 and had 2 routers on one side and 2 routers on the other side that is only 2 sets of macs at layer 2 and one router set would be on one port and the other router set would be on the other port for each connection the routers generate. 

Port sharing never just looks at packet count or bandwidth so that is why you can have larger amounts of traffic on a single interface of a port share group. 

Mcast and broadcast frames typical have a common destination mac so that traffic is even less likely to be spread across more than one port for each of the flows being generated be your routers or clients. 

People think of port sharing as a way to get more bandwidth but it truely depends on what type of Ethernet traffic you are pushing across your switch. 

These are my findings from being a layer 2 network providor using extreme for 20 plus years.  They may not have all the true technical info of how the switch choose to forward traffic when you have a LACP port group and share enabled, but this is my understanding of how it works and for sure my real world experience.  We use L3/L4 on all our port shares.  

One other issue you will run into is when you hand off to another vendor.. in to out is what you control.. so traffic from your switch to another vendor .. you can't influence what they send you.  Most vendors today use some sort of L3/L4 ... 
  

 
Photo of Keith Olmstead

Keith Olmstead

  • 116 Points 100 badge 2x thumb
So if there is a 2 port share and you have 2 sets of source and destination macs one set would be on one port and the next set would be on the other port.
Thanks, so am I right by thinking that our two shared fiber legs are just duplicating the forwarding database and not splitting source and destination macs.
Photo of EtherMAN

EtherMAN, Embassador

  • 6,960 Points 5k badge 2x thumb
yes and no.... if you only had a single mac on each side of a 2 port share and you had L3/L4 configured you would still see those individual data sessions being split back and forth... If it is a stream such as you would see with a IPTV mcast it will stay on the port it landed on till the stream is pruned and not wanted... The days of this pack based sharing on switches is over.  You can try the custom setup and changing the Hash like Stephane says... If you are really looking to load balance and not load share.  Load balance is typically a layer 3 function and or done by dedicated piece of hardware designed to load balance the traffic equally across multiple paths vs load share which is single traffic flows are sent down a group of shared connection,  

You kind of have to go back to basic network knowledge and understand what is on both sides of your connection and how they request and exchange traffic with each other.  
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 13,346 Points 10k badge 2x thumb
I'd change the hashing algorithm to custom and play with the hash to see what works better for you: XOR, CRC16 or CRC32.