topic RE: Load-balance problem using VPLS as transport in ExtremeSwitching (EXOS/Switch Engine)

Load-balance problem using VPLS as transport

Anatoly_Machekh — Wed, 09 Nov 2016 19:46:00 GMT

Hi, everyone!

Here is lab network to test balancing on client ports on Extreme Network switches (X460 in lab, but we have the same problem with X670-G2 in production).
We are interested in sharing group 21 (ports 21, 22) on X460-1.

From Agilent (IP tester) IP traffic goes to Lab-M01 to 18 different VRFs (1001-1009 and 2001-2009). Lab-M01 sends out traffic in VRF 1001 through 1009 with one MPLS label, in VRF 2001 through 2009 with two MPLS labels. When we configure X460-1 and X460-2 as simple L2 switches with one vlan from sharing group 21 on X460-1 to sharing group 21 on X460-2, then balancing works.

1 MPLS label:

X460-1.8 # sh port 21,22,28 utilization packets
Port Link Rx Peak Rx Tx Peak Tx
State pkts/sec pkts/sec pkts/sec pkts/sec
================================================================================
21 A 0 1 6632 8034
22 A 0 0 13238 16037
28 A 19857 23909 0 1
================================================================================
2 MPLS labels:

X460-1.10 # sh port 21,22,28 utilization packets
Port Link Rx Peak Rx Tx Peak Tx
State pkts/sec pkts/sec pkts/sec pkts/sec
================================================================================
21 A 2 3 8725 10678
22 A 0 1 13118 13208
28 A 23657 23731 3 3
================================================================================
The problem is that when we configure VPLS on X460-1 and X460-2 to pass the same traffic balancing stops working. All traffic goes to one port (either 21 or 22 depending...).
What can we do to make balancing work with VPLS?

Thanks,
Anatoly

RE: Load-balance problem using VPLS as transport

Stephane_Grosj1 — Wed, 09 Nov 2016 20:02:00 GMT

This is a LAG, correct? What config is on it (algo, hashing)? I'd recommend custom there. If not good enough, since 15.4 we have a feature - LSP Load Sharing - (for VPLS and RSVP-TE) to distribute the traffic more efficiently, using more LSPs (p.1123 of 22.1 User Guide).

RE: Load-balance problem using VPLS as transport

Anatoly_Machekh — Wed, 09 Nov 2016 20:08:00 GMT

Here is information on X460:
X460-1.1 # sh conf | i sharing enable sharing 21 grouping 21-22 algorithm address-based custom lacp
X460-1.2 # sho sharing
Load Sharing Monitor
Config Current Agg Min Ld Share Flags Ld Share Agg Link Link Up
Master Master Control Active Algorithm Group Mbr State Transitions
================================================================================
21 21 LACP 1 custom A 21 Y A 2
custom 22 Y A 0
================================================================================
Link State: A-Active, D-Disabled, R-Ready, NP-Port not present, L-Loopback
Minimum Active: (<) Group is down. # active links less than configured minimum
Load Sharing Algorithm: (L2) Layer 2 address based, (L3) Layer 3 address based
(L3_L4) Layer 3 address and Layer 4 port based
(custom) User-selected address-based configuration
Custom Algorithm Configuration: ipv4 L3-and-L4, xor
Distribution Mode Flags:
A - All: Distribute to all members
L - Local Slot: Distribute to members local to ingress slot
P - Port Lists: Distribute to per-slot configurable subset of members
Number of load sharing trunks: 1

RE: Load-balance problem using VPLS as transport

Stephane_Grosj1 — Wed, 09 Nov 2016 20:19:00 GMT

Then configure LSP load sharing.

RE: Load-balance problem using VPLS as transport

Anatoly_Machekh — Wed, 09 Nov 2016 20:25:00 GMT

In our lab test we have one LDP LSP for VPLS. It goes from X460-2 to X460-1. It does not go through sharing group 21 to customer equipment.

X460-1.1 # sho mpls ldp lsp Prefix Adv Label Peer Label Next Hop VLAN
10.0.0.1/32 0x00434 -- -- LB1
10.0.0.2/32 -- 0x00434 172.22.0.2 VLAN_12

RE: Load-balance problem using VPLS as transport

Korsovsky__Kons — Wed, 09 Nov 2016 20:25:00 GMT

Hi Anatoly,
we had discussion within service case but this is just to share with community.

As Stephane said we can utilize LSP load sharing feature. Here are examples from my lab:

Test 1
Description: single VPLS tunnel without LSP load sharing

Sharing algorithm cannot distribute packets across sharing group member ports because packet headers are almost the same.

Test 2
Description: single VPLS tunnel with LSP load sharing

In this case packets from test-flows are shared between several LSPs and it makes MPLS labels to be different for the different packets from the same VPLS tunnel:
X670G2-48X[L5:U32].14 # show mpls statistics l2vpn

L2VPN Name
Peer IP/LSP Flags RxPackets RxBytes TxPackets TxBytes
----------------------------------------------------------------------------------------------------------
VPLS_VPN
10.0.0.35 U 2920 227552 15099705 1298574028
LSP1 + 360 28720 1886864 162270218
LSP2 + 370 29760 1885514 162153946
LSP3 + 342 28200 1885072 162116106
LSP4 + 350 28562 1888660 162424674
LSP5 + 381 27602 1887981 162366366
LSP6 + 372 28002 1887950 162363700
LSP7 + 375 27200 1888686 162426910
LSP8 + 370 29506 1888978 162452108
Here is configuration example for this setup:
# # Module mpls configuration. # configure mpls add vlan "lo0" enable mpls vlan "lo0" enable mpls rsvp-te vlan "lo0" enable mpls ldp vlan "lo0" configure mpls add vlan "v3233" enable mpls vlan "v3233" enable mpls rsvp-te vlan "v3233" enable mpls ldp vlan "v3233" create mpls rsvp-te path "lsp-path" create mpls rsvp-te lsp "LSP1" destination 10.0.0.35 create mpls rsvp-te lsp "LSP2" destination 10.0.0.35 create mpls rsvp-te lsp "LSP3" destination 10.0.0.35 create mpls rsvp-te lsp "LSP4" destination 10.0.0.35 create mpls rsvp-te lsp "LSP5" destination 10.0.0.35 create mpls rsvp-te lsp "LSP6" destination 10.0.0.35 create mpls rsvp-te lsp "LSP7" destination 10.0.0.35 create mpls rsvp-te lsp "LSP8" destination 10.0.0.35 configure mpls rsvp-te lsp "LSP1" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP2" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP3" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP4" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP5" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP6" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP7" add path "lsp-path" primary configure mpls rsvp-te lsp "LSP8" add path "lsp-path" primary configure mpls lsr-id 10.0.0.32 enable mpls protocol ldp enable mpls protocol rsvp-te enable mpls create l2vpn vpls VPLS_VPN fec-id-type pseudo-wire 111 configure l2vpn vpls VPLS_VPN add service vlan v1000 enable l2vpn sharing configure l2vpn vpls VPLS_VPN add peer 10.0.0.35 core full-mesh configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP1 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP2 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP3 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP4 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP5 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP6 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP7 configure l2vpn vpls VPLS_VPN peer 10.0.0.35 add mpls lsp LSP8 As you can see traffic can be well distributed across links when sharing algorithm can get packets with different MPLS labels.

Test 3
Description: single LSP and multiply source ports involved in sharing

In this example sharing algorithm can use information about source ports to better distribute traffic across ports in sharing group.

RE: Load-balance problem using VPLS as transport

Renato_Santos1 — Mon, 29 Jul 2019 22:29:00 GMT

Please, I followed the instructions above but didn't have success. Is it possible you check my configuration bellow?:

Topollogy:

Router 1:
ATG_TESTE.19 # sh conf
#
# Module devmgr configuration.
#
configure snmp sysName "ATG_TESTE"
configure snmp sysContact "support@extremenetworks.com, +1 888 257 3000"
configure sys-recovery-level switch reset

#
# Module vlan configuration.
#
configure vlan default delete ports all
configure vr VR-Default delete ports 1-64
configure vr VR-Default add ports 1-64
configure vlan default delete ports 1-64
configure jumbo-frame-size size 9198
configure sharing address-based custom hash-algorithm crc-32 lower
create vlan "L3_ATG_SFO"
configure vlan L3_ATG_SFO tag 50
create vlan "LPBK_MPLS"
configure vlan LPBK_MPLS tag 147
enable loopback-mode vlan LPBK_MPLS
create vlan "MPLS_balance"
configure vlan MPLS_balance tag 77
enable jumbo-frame ports 1
configure ports 1 display-string 1.MPLS_in
configure ports 1 auto on speed 1000
enable jumbo-frame ports 47
configure ports 47 display-string 47.MPLS_ATG_SFO
configure ports 47 auto off speed 1000 duplex full
enable jumbo-frame ports 48
configure ports 48 display-string 48.MPLS_ATG_SFO
configure ports 48 auto off speed 1000 duplex full
enable sharing 47 grouping 47-48 algorithm address-based custom lacp
configure vlan L3_ATG_SFO add ports 47 untagged
configure vlan MPLS_balance add ports 47 tagged
configure vlan MPLS_balance add ports 1 untagged
configure vlan Mgmt ipaddress 20.2.2.200 255.255.0.0
configure vlan MPLS_balance ipaddress 10.10.10.200 255.255.255.0
configure vlan L3_ATG_SFO ipaddress 31.0.23.1 255.255.255.252
enable ipforwarding vlan L3_ATG_SFO
configure vlan LPBK_MPLS ipaddress 21.41.47.1 255.255.255.255
enable ipforwarding vlan LPBK_MPLS
configure ip-mtu 1550 vlan LPBK_MPLS

#
# Module rtmgr configuration.
#
enable iproute sharing vr VR-Default

#
# Module mcmgr configuration.
#
disable igmp snooping vlan "L3_ATG_SFO"
disable igmp snooping vlan "MPLS_balance"

#
# Module mpls configuration.
#
configure mpls add vlan "L3_ATG_SFO"
enable mpls vlan "L3_ATG_SFO"
enable mpls rsvp-te vlan "L3_ATG_SFO"
enable mpls ldp vlan "L3_ATG_SFO"
configure mpls add vlan "LPBK_MPLS"
enable mpls rsvp-te vlan "LPBK_MPLS"
enable mpls ldp vlan "LPBK_MPLS"
enable mpls php vlan "LPBK_MPLS"
create mpls rsvp-te path "path-link"
configure mpls rsvp-te path path-link add ero include 31.0.23.2/32 strict order 10
create mpls rsvp-te lsp "lsp1" destination 21.41.23.1
create mpls rsvp-te lsp "lsp2" destination 21.41.23.1
create mpls rsvp-te lsp "lsp3" destination 21.41.23.1
create mpls rsvp-te lsp "lsp4" destination 21.41.23.1
configure mpls rsvp-te lsp "lsp1" add path "path-link" primary
configure mpls rsvp-te lsp "lsp2" add path "path-link" primary
configure mpls rsvp-te lsp "lsp3" add path "path-link" primary
configure mpls rsvp-te lsp "lsp4" add path "path-link" primary
configure mpls lsr-id 21.41.47.1
enable mpls protocol ldp
enable mpls protocol rsvp-te
enable mpls
create l2vpn vpls vpls-v77 fec-id-type pseudo-wire 77
configure l2vpn vpls vpls-v77 add service vlan MPLS_balance
configure l2vpn vpls vpls-v77 mtu 9198
enable l2vpn sharing
configure l2vpn vpls vpls-v77 add peer 21.41.23.1 core full-mesh
configure l2vpn vpls vpls-v77 peer 21.41.23.1 add mpls lsp lsp1
configure l2vpn vpls vpls-v77 peer 21.41.23.1 add mpls lsp lsp2
configure l2vpn vpls vpls-v77 peer 21.41.23.1 add mpls lsp lsp3
configure l2vpn vpls vpls-v77 peer 21.41.23.1 add mpls lsp lsp4

#
# Module ospf configuration.
#
configure ospf routerid 21.41.47.1
enable ospf mpls-next-hop
enable ospf
create ospf area 0.0.0.40
configure ospf add vlan L3_ATG_SFO area 0.0.0.40 link-type point-to-point
configure ospf vlan L3_ATG_SFO cost 20
configure ospf vlan L3_ATG_SFO priority 0
configure ospf add vlan LPBK_MPLS area 0.0.0.40
configure ospf vlan LPBK_MPLS priority 0

#
# Module ospfv3 configuration.
#
configure ospfv3 spf-hold-time 10

#
# Module thttpd configuration.
#
configure ssl certificate hash-algorithm sha512

Router 2 (the similar to the first)

RE: Load-balance problem using VPLS as transport

Renato_Santos1 — Thu, 01 Aug 2019 23:26:00 GMT

In the example that Korsovsky, Konstantin did, how kind of sharing was used?

L2 or L3_L4 or port??