topic RE: ERPS subring problem. in ExtremeSwitching (EXOS/Switch Engine)

ERPS subring problem.

Nikolay_Chernya — Mon, 30 Nov 2015 19:46:00 GMT

Hi Extreme Community.We have some experience with EAPS, now trying to make some lab test with ERPS and various vendors to push ERPS in to production.

Currently we are observing following problem:
When main ring broken, subring RPL owner does not unblocks RPL port.

Here is our lab.

SW1 config:
#VLANS
create vlan "MNG"
configure vlan MNG tag 9
configure vlan MNG add ports 1-3 tagged
configure vlan MNG ipaddress 10.10.10.111 255.255.255.0
create vlan "control_main"configure vlan control_main tag 305
configure vlan control_main add ports 2-3 tagged
configure vlan control_sub tag 304configure vlan control_sub add ports 1-3 tagged

#ERPS MAIN RING
create erps test
configure erps test subring-mode virtualChannel
configure erps test add control vlan control_main
configure erps test ring-port east 3
configure erps test ring-port west 2
configure erps test protection-port 2
configure erps test revert enable
configure erps test timer wait-to-restore 60000
configure erps test add protected vlan MNG
configure erps test add protected vlan control_sub
configure erps test add sub-ring test-sub
enable erps test

#ERPS SUB-RING
create erps test-sub
configure erps test-sub subring-mode virtualChannel
configure erps test-sub add control vlan control_sub
configure erps test-sub ring-port east 1
configure erps test-sub revert enable
configure erps test-sub timer wait-to-restore 60000
configure erps test-sub add protected vlan MNG

#SW2 config:

#MST configuration (protected vlans, mst disabled)
spanning-tree mst configuration
instance 0 vlan 1-8;10-303;305-4094
instance 1 vlan 9;304

#ERPS
erps-ring test
open-ring
raps-virtual-channel with
erps-instance 1
wtr-timer 1
raps-mel 0
rpl port1 owner
protected-instance 1
control-vlan 304

#PORTS
Interface Ethernet1/0/25
switchport mode trunk
erps-ring test port0
!
Interface Ethernet1/0/26
switchport mode trunk
erps-ring test port1

SW3 config:
#VLANS
create vlan "MNG"configure vlan MNG tag 9
configure vlan MNG add ports 31-33 tagged
configure vlan MNG ipaddress 10.10.10.114 255.255.255.0
create vlan "control_main"
configure vlan control_main tag 305
create vlan "control_sub"
configure vlan control_sub tag 304
configure vlan control_main add ports 32-33 tagged
configure vlan control_sub add ports 31-33 tagged

#ERPS MAIN RING
create erps testconfigure erps test subring-mode virtualChannel
configure erps test add control vlan control_main
configure erps test ring-port east 32
configure erps test ring-port west 33
configure erps test revert enable
configure erps test neighbor-port 32
configure erps test timer wait-to-restore 60000
configure erps test add protected vlan MNG
configure erps test add protected vlan control_sub
configure erps test add sub-ring test-sub
enable erps test

#ERPS SUB-RING
create erps test-sub
configure erps test-sub subring-mode virtualChannel
configure erps test-sub add control vlan control_sub
configure erps test-sub ring-port east 31
configure erps test-sub revert enable
configure erps test-sub neighbor-port 31
configure erps test-sub timer wait-to-restore 60000
configure erps test-sub add protected vlan MNG
enable erps test-sub

Ok, all configs are done, let's see what is going on:

#SW1 main ERPS:
sw1# sh erps test
Name: test
Operational State: Idle Node Type: RPL Owner, Revertive
Configured State : Enabled

East Ring Port : 3 MepId: ---- Remote MepId: ---- Status: Unblocked
West Ring Port : +2 MepId: ---- Remote MepId: ---- Status: Blocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
Number of Protected Vlans: 2
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW1 subring erps:
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: InterConnection Node
Configured State : Enabled

East Ring Port : 1 MepId: ---- Remote MepId: ---- Status: Unblocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#On sw2:
#sh erps instance
ERPS Ring: test
Instance: 1
Description: -
Protected Instance: 1
Revertive mode: revertive
R-APS MEL: 0
R-APS Virtual-Channel: with
Control Vlan: 304
Ring ID: 1
Guard Timer(10ms): 50
Holdoff Timer(seconds): 0
WTR Timer(min): 1
-------------------------------------------
Port Role Port-Status
-------------------------------------------
Port0 common forwarding
Port1 RPL owner blocked

sh erps ring test
R: RPL Owner
N: RPL Neighbour
C: Common Node
-----------------------------------------
Version: ITU-T G.8032v2
R-APS ring topology: open-ring
R-APS Virtual-Channel: with
Port0: Ethernet1/0/25
Failure-detect type: physical-link
Port1: Ethernet1/0/26
Failure-detect type: physical-link
--------------------------------------------------------------------------------------------------------------------------------
Instance Contral Protected WTR_Timer Guard_Timer Holdoff_Timer
ID Vlan Instance (min) (10ms) (second) Port0 Port1
--------------------------------------------------------------------------------------------------------------------------------
1 304 1 1 50 0 C R

#on SW3:
#MAIN ERPS:
sh erps test
Name: test
Operational State: Idle Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^32 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : 33 MepId: ---- Remote MepId: ---- Status: Unblocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
Number of Protected Vlans: 2
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#subring ERPS
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^31 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

And now, we got a problem. When both links in main ERPS ring fails, RPL owner in subring does not opens RPL link.

#SW1
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: InterConnection Node
Configured State : Enabled

East Ring Port : 1 MepId: ---- Remote MepId: ---- Status: Unblocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW2
sh erps instanceERPS Ring: test
Instance: 1
Description: -
Protected Instance: 1
Revertive mode: revertive
R-APS MEL: 0
R-APS Virtual-Channel: with
Control Vlan: 304
Ring ID: 1
Guard Timer(10ms): 50
Holdoff Timer(seconds): 0
WTR Timer(min): 1
-------------------------------------------
Port Role Port-Status
-------------------------------------------
Port0 common forwarding
Port1 RPL owner blocked

#SW3
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^31 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

Any help will be highly appreciated.

RE: ERPS subring problem.

Nikolay_Chernya — Mon, 30 Nov 2015 21:33:00 GMT

Config is similar to:
https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-configure-a-ERPS-ring-with-a-sub-ring-attached/

We have also tested this config in UNetLab with extreme summit switches only, with same simptoms.

RE: ERPS subring problem.

Anton_Nekhorosh — Tue, 01 Dec 2015 23:59:00 GMT

Anybody has no ideas?!

RE: ERPS subring problem.

dflouret — Wed, 02 Dec 2015 02:22:00 GMT

Nikolai, Anton,

From the EXOS User Guide v16.1:
G.8032 Version 2 (pg. 1086)
The concept of sub-rings is introduced to add multiple rings to the main ring. A sub-ring is an incomplete ring that completes its path through the main ring or other sub-rings. The control path for the sub-ring completes either through the implementation of a virtual channel, or by changing the flow of control packets in the sub-rings. Virtual channels are supported through the use of the sub-rings control channel being configured as a data VLAN in the main ring.

Multiple Failure Protection using CFM (pg. 1087)
You can use CFM UP-MEP support in the sub-ring to provide protection against multiple failures in the main ring. Configure an UP-MEP on the interconnected nodes, where a segmentation of the main ring results in the UP-MEP notifying the sub-ring of a failure. This causes the sub-ring to open its RPL and place the interconnected node in manual switch. This is done to avoid a super-loop during recovery of the main ring. This implementation is as directed in Appendix X.3 of the G.8032 standard.

Let me "translate" this.

When a failure occurs in an ERPS ring, the nodes that own the ERPS ports involved in the failure will send CFM R-‐APS messages to inform of the failure, which makes all nodes in the ring to flush their tables and the owner of the RPL port to close the RPL port. In your diagram, Nikolai, if the left link between SW1 and SW3 failed SW3 would inform of the failure through the main ring's control VLAN. Upon receiving the notification, SW1 would close the RPL port and reestablish connectivity to SW3.

What happens to the sub-ring when the main ring fails? Nothing. Since the failure does not involve any of its ports or links, no message is transmitted through the sub-ring's control vlan and no action is taken by the sub-ring nodes. The main ring "heals" itself and maintains the connectivity of SW2 with SW3, as indicated in the first paragraph quoted from the User Guide.

Similarly, if the link between SW1 and SW2 failed, SW1would inform SW2 of the failure through the sub-ring's control VLAN (which happens to go through the main ring). Upon receiving the notification, SW2 would close the RPL port and reestablish connectivity to SW1 and SW3.

Now, the problem comes when we have MULTIPLE failures in the main ring, making it unable to maintain the connectivity between the nodes of the sub-ring. The sub-ring will not detect the failures, as explained previously, because they don't involve ports belonging to the sub-ring. That is why you need to configure CFM, as indicated in the second paragraph quoted from the User Guide.

Now, if you're familiar with EAPS, you might wonder why you need to configure CFM. Well, EAPS includes a mechanism where the Master node send a "health check" message through the control vlan to check that there's connectivity in the ring. This is slow but, in case a failure goes undetected by the other nodes in the ring, the Master will eventually heal the ring. In ERPS the RPL owner doesn't do that. ERPS relies on the configuration of CFM Connectivity Check Messages (CCM) to detect failures not reported by the ring nodes.

To make your configuration work you will need to configure CFM Maintenance End Points (MEPs). There's an sample configuration in the EXOS User Guide v16.1(page 1094), which is almost identical to yours, except that it has 4 switches, instead of 3.

Here's a How-To Guide that explains the basics of CFM R-APS messaging and error detection and notification in a single ERPS ring:
https://www.dropbox.com/s/l5m3fuirlsbmrru/SVC%20Tech%20MB%20Tech%20Guide%20-%20ERPSv1%20%28G.8032v1%...

RE: ERPS subring problem.

Nikolay_Chernya — Wed, 02 Dec 2015 18:42:00 GMT

Thank you Daniel, yours answer is very helpfull.
If you have and sample config for UP meps similar to our config, it will be great to see them here.

RE: ERPS subring problem.

dflouret — Wed, 02 Dec 2015 23:21:00 GMT

Nikolai,

I've just done a "quick and dirty" config using EXOS-VM switches, and here's my config:

# Configuration for SW 1
# CFM Up MEP Configuration for ERPS sub-ring
create cfm domain string sr1 md-level 3
configure cfm domain sr1 add association string MDlevel1 vlan cfmVlan
configure cfm domain sr1 association MDlevel1 ports 1 add end-point up 100
configure cfm domain sr1 association MDlevel1 ports 1 end-point up add group S2
configure cfm group S2 add rmep 200
configure erps sub_ring cfm protection primary add group S2

# Configuration for SW 2
# CFM Up MEP configuration for ERPS sub-ring
create cfm domain string sr1 md-level 3
configure cfm domain sr1 add association string MDlevel1 vlan cfmVlan
configure cfm domain sr1 association MDlevel1 ports 31 add end-point up 200
configure cfm domain sr1 association MDlevel1 ports 31 end-point up add group S1
configure cfm group S1 add rmep 100
configure erps sub_ring cfm protection primary add group S1

cfmVlan is the vlan through which the CFM messages will be sent. This vlan CAN'T BE a ring control vlan.

The following log messages were generated in SW1 when both links between SW1 and SW2 failed:

12/01/2015 21:15:46.85 : ERPS ring sub_ring state changed from old state Idle to new state Protection
12/01/2015 21:15:46.85 : Remote End-Point 200/00:01:30:DF:05:02 expired on up end-point, domain "sr1", ma "MDlevel1", port 3
12/01/2015 21:15:44.15 : Port 2 link down
12/01/2015 21:15:42.16 : Port 1 link down
12/01/2015 21:15:42.06 : ERPS ring main_ring state changed from old state Idle to new state Protection

These appeared in SW2:

12/01/2015 21:15:45.00 : Maintenance End-Point 200 Error Not recv CCM from RMEP occurred on up end-point, domain "sr1", ma "MDlevel1", port 3
12/01/2015 21:15:42.50 : ERPS ring sub_ring state changed from old state Idle to new state Protection
12/01/2015 21:15:42.50 : Remote End-Point 100/00:01:30:DF:05:01 expired on up end-point, domain "sr1", ma "MDlevel1", port 3
12/01/2015 21:15:39.45 : Port 2 link down
12/01/2015 21:15:37.43 : ERPS ring main_ring state changed from old state Idle to new state Protection
12/01/2015 21:15:37.43 : Port 1 link down

And these in SW3:

12/01/2015 21:15:09.35 : ERPS ring sub_ring state changed from old state Idle to new state Protection

RE: ERPS subring problem.

Anton_Nekhorosh — Wed, 02 Dec 2015 23:21:00 GMT

Daniel, below the same config don't work, may be you have other version ExtremeXOS? our version is 15.4.1.3 v1541b3-patch1-13
Something can another?

RE: ERPS subring problem.

Nikolay_Chernya — Wed, 02 Dec 2015 23:21:00 GMT

We have updated to recomended verison few days ago - ExtremeXOS version 15.6.3.1 v1563b1-patch1-5, with same result.All output posted in this thread is from 15.6.3.1.

RE: ERPS subring problem.

Nikolay_Chernya — Wed, 02 Dec 2015 23:31:00 GMT

I have reviewed RFC appendix and configured UP meps on subring ports on interconnected nodes.
Now, SW2 get signalled when multiply failure on main ring occurs, and opens RPL, but ports on interconnected nodes facing sub-ring goes to blocked state.

added configuration:

#SW1 VLAN
create vlan "subring-mep"
configure vlan subring-mep tag 306
configure vlan subring-mep add ports 1-3 tagged

#SW1 CFM:
create cfm domain string "test" md-level 6configure cfm domain "test" add association string "level6" vlan "subring-mep"
configure cfm domain "test" association "level6" ports 1 add end-point up 601
configure cfm domain "test" association "level6" ports 1 end-point up add group "sw1-sw3"
configure cfm group "sw1-sw3" add rmep 501
#SW1 ERPS
configure erps test-sub cfm protection primary add group sw1-sw3
configure erps test add protected vlan subring-mep

#SW3 VLAN
create vlan "subring-mep"
configure vlan subring-mep tag 306
configure vlan subring-mep add ports 31-33 tagged

#SW3 CFM:
create cfm domain string "test" md-level 6configure cfm domain "test" add association string "level6" vlan "subring-mep"
configure cfm domain "test" association "level6" ports 31 add end-point up 501
configure cfm domain "test" association "level6" ports 31 end-point up add group "sw3-sw1"
configure cfm group "sw3-sw1" add rmep 601

#SW3 ERPS
configure erps test-sub cfm protection secondary add group sw3-sw1
configure erps test add protected vlan subring-mep

STATES:

#SW1 CFM state when all ports are enabled:
sh cfmDomain: "test", MD Level: 6
Association: "level6", Destination MAC Type: Multicast, VLAN "subring-mep" with 3 cfm ports
Transmit Interval: 1000 ms, Type : IEEE 802.1ag Maintenance Association
port 1; Up End Point, mepid: 601, transmit-interval: 1000 ms (from association)
MEP State: Enabled, CCM Message: Enabled, Send SenderId TLV: Disabled

Faulting State : No
Last Faulting State Change : Wed Dec 2 19:21:28 2015

MEP Error Defects : None
Port Status : Up port 2; Intermediate Point ( Dynamic )
port 3; Intermediate Point ( Dynamic )

Total Number of Domain : 1
Total Number of Association : 1
Total Number of Up MEP : 1
Total Number of Down MEP : 0
Total Number of MIP : 2
Total Number of CFM port : 3
Total Number of SW MEP : 1
Total Number of HW MEP : 0
Total Number of VPLS MIP(Static/Up): 0 / 0

================================================================================
MEP Error Defect Types:
Remote : Not receiving CCMs from Remote MEP
Error : Erroneous CCM received
XCON : Cross-connect CCM received
RDI : Remote Defect Indication sent by some MEP

#SW1 MAIN ring:
sh erps test
Name: test
Operational State: Idle Node Type: RPL Owner, Revertive
Configured State : Enabled

East Ring Port : 3 MepId: ---- Remote MepId: ---- Status: Unblocked
West Ring Port : +2 MepId: ---- Remote MepId: ---- Status: Blocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
subring-mep 306
Number of Protected Vlans: 3
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW1 ERPS SUB-RING
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: InterConnection Node
Configured State : Enabled

East Ring Port : 1 MepId: ---- Remote MepId: ---- Status: Unblocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Subring Protection Group: sw1-sw3

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW2 ERPS
sh erps instanceERPS Ring: test
Instance: 1
Description: -
Protected Instance: 1
Revertive mode: revertive
R-APS MEL: 0
R-APS Virtual-Channel: with
Control Vlan: 304
Ring ID: 1
Guard Timer(10ms): 50
Holdoff Timer(seconds): 0
WTR Timer(min): 1
-------------------------------------------
Port Role Port-Status
-------------------------------------------
Port0 common forwarding
Port1 RPL owner blocked

#SW3 CFM
sh cfmDomain: "test", MD Level: 6
Association: "level6", Destination MAC Type: Multicast, VLAN "subring-mep" with 3 cfm ports
Transmit Interval: 1000 ms, Type : IEEE 802.1ag Maintenance Association
port 31; Up End Point, mepid: 501, transmit-interval: 1000 ms (from association)
MEP State: Enabled, CCM Message: Enabled, Send SenderId TLV: Disabled

Faulting State : No
Last Faulting State Change : Wed Dec 2 16:20:51 2015

MEP Error Defects : None
Port Status : Up port 32; Intermediate Point ( Dynamic )
port 33; Intermediate Point ( Dynamic )

Total Number of Domain : 1
Total Number of Association : 1
Total Number of Up MEP : 1
Total Number of Down MEP : 0
Total Number of MIP : 2
Total Number of CFM port : 3
Total Number of SW MEP : 1
Total Number of HW MEP : 0
Total Number of VPLS MIP(Static/Up): 0 / 0

================================================================================
MEP Error Defect Types:
Remote : Not receiving CCMs from Remote MEP
Error : Erroneous CCM received
XCON : Cross-connect CCM received
RDI : Remote Defect Indication sent by some MEP

#SW3 ERPS MAIN
sh erps test
Name: test
Operational State: Idle Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^32 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : 33 MepId: ---- Remote MepId: ---- Status: Unblocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
subring-mep 306
Number of Protected Vlans: 3
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW3 ERPS SUB-RING
sh erps "test-sub"
Name: test-sub
Operational State: Idle Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^31 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Subring Protection Group: sw3-sw1

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

##############################
Now, if we disable both ports in main ring, we can see following:

#SW1 CFM
sh cfmDomain: "test", MD Level: 6
Association: "level6", Destination MAC Type: Multicast, VLAN "subring-mep" with 3 cfm ports
Transmit Interval: 1000 ms, Type : IEEE 802.1ag Maintenance Association
port 1; Up End Point, mepid: 601, transmit-interval: 1000 ms (from association)
MEP State: Enabled, CCM Message: Enabled, Send SenderId TLV: Disabled

Faulting State : Yes
Last Faulting State Change : Wed Dec 2 19:25:52 2015

MEP Error Defects : Remote
Port Status : Up port 2; Intermediate Point ( Dynamic )
port 3; Intermediate Point ( Dynamic )

Total Number of Domain : 1
Total Number of Association : 1
Total Number of Up MEP : 1
Total Number of Down MEP : 0
Total Number of MIP : 2
Total Number of CFM port : 3
Total Number of SW MEP : 1
Total Number of HW MEP : 0
Total Number of VPLS MIP(Static/Up): 0 / 0

================================================================================
MEP Error Defect Types:
Remote : Not receiving CCMs from Remote MEP
Error : Erroneous CCM received
XCON : Cross-connect CCM received
RDI : Remote Defect Indication sent by some MEP

#SW1 ERPS MAIN
sh erps test
Name: test
Operational State: Protection Node Type: RPL Owner, Revertive
Configured State : Enabled

East Ring Port : 3 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : +2 MepId: ---- Remote MepId: ---- Status: Blocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
subring-mep 306
Number of Protected Vlans: 3
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW1 ERPS SUB-RING
sh erps "test-sub"
Name: test-sub
Operational State: Protection Node Type: InterConnection Node
Configured State : Enabled

East Ring Port : 1 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Subring Protection Group: sw1-sw3

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW2
sh erps instanceERPS Ring: test
Instance: 1
Description: -
Protected Instance: 1
Revertive mode: revertive
R-APS MEL: 0
R-APS Virtual-Channel: with
Control Vlan: 304
Ring ID: 1
Guard Timer(10ms): 50
Holdoff Timer(seconds): 0
WTR Timer(min): 1
-------------------------------------------
Port Role Port-Status
-------------------------------------------
Port0 common forwarding
Port1 RPL owner forwarding

#SW3 CFM
sh cfmDomain: "test", MD Level: 6
Association: "level6", Destination MAC Type: Multicast, VLAN "subring-mep" with 3 cfm ports
Transmit Interval: 1000 ms, Type : IEEE 802.1ag Maintenance Association
port 31; Up End Point, mepid: 501, transmit-interval: 1000 ms (from association)
MEP State: Enabled, CCM Message: Enabled, Send SenderId TLV: Disabled

Faulting State : Yes
Last Faulting State Change : Wed Dec 2 16:25:14 2015

MEP Error Defects : Remote
Port Status : Up port 32; Intermediate Point ( Dynamic )
port 33; Intermediate Point ( Dynamic )

Total Number of Domain : 1
Total Number of Association : 1
Total Number of Up MEP : 1
Total Number of Down MEP : 0
Total Number of MIP : 2
Total Number of CFM port : 3
Total Number of SW MEP : 1
Total Number of HW MEP : 0
Total Number of VPLS MIP(Static/Up): 0 / 0

================================================================================
MEP Error Defect Types:
Remote : Not receiving CCMs from Remote MEP
Error : Erroneous CCM received
XCON : Cross-connect CCM received
RDI : Remote Defect Indication sent by some MEP

#SW3 ERPS MAIN
sh erps test
Name: test
Operational State: Protection Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^32 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : 33 MepId: ---- Remote MepId: ---- Status: Blocked

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_main VID:305
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Major ring has following Sub-Ring(s):
Sub-Ring Name : test-sub
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
control_sub 304
subring-mep 306
Number of Protected Vlans: 3
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

#SW3 SUB-RING
sh erps "test-sub"
Name: test-sub
Operational State: Protection Node Type: RPL Neighbor
Configured State : Enabled

East Ring Port : ^31 MepId: ---- Remote MepId: ---- Status: Blocked
West Ring Port : ----- MepId: ---- Remote MepId: ---- Status: ------

Subring Protection Group: sw3-sw1

Periodic timer interval: 5000 millisec (Enabled)
Hold-off timer interval: 0 millisec (Enabled)
Guard timer interval : 500 millisec (Enabled)
WTB timer interval : 5500 millisec (Enabled)
WTR timer interval : 60000 millisec (Enabled)

Ring MD Level : ----
Notify Topology Change : -------
Subring Mode : Virtual Channel

ERPS Control Vlan: control_sub VID:304
Topology Change Propogation List: None
Topology Change Propogation : Disabled
ERPS Ring's Main-Ring : test
ERPS Ring has following Protected Vlan(s):
Vlan Name VID
MNG 9
Number of Protected Vlans: 1
(+) RPL Protection Port, (^) RPL Neighbor Port
(f) Force Switch Port, (m) Manual Switch Port

So, both ports on interconnected nodes facing subring are blocked.I have tried setting primary and secondary protection path on both nodes with no luck.
Looks like i just misunderstanding something here.

Any help will be appreciated, thanks in advance.

RE: ERPS subring problem.

Nikolay_Chernya — Thu, 03 Dec 2015 01:27:00 GMT

Daniel, looks like we wrote our posts simultaneously.I have done all this configuration today, but got problems on interconnected nodes - ports facing sub-ring blocked.
All output you can check in my previous post.
The only difference in our configurations - i have added cfm-vlan to protected vlan in main ring to avoid loop.
Also, you wrote configuration for SW2, in my diagram it should be SW3 according to RFC appendix.
Any thoughts will be helpful.

RE: ERPS subring problem.

Nikolay_Chernya — Fri, 04 Dec 2015 16:05:00 GMT

I have replaced Qtech 8200 with Juniper EX4500 with same result.

RE: ERPS subring problem.

dflouret — Fri, 04 Dec 2015 19:15:00 GMT

Nikolay, I suggest that you open a case with GTAC so they can take a look at what's going on here.

RE: ERPS subring problem.

Nikolay_Chernya — Sat, 19 Dec 2015 20:38:00 GMT

Just for the record.
We have discovered an unexpected behavior with ERPS, this is a bug i think.
If any port participating in ERPS formed as a sharing, and config master disabled while other ports in sharing are in service, deleting any ERPS protecting vlan or removing vlan from ring ports causing ERPS to switch to protection.

RE: ERPS subring problem.

Luca_Spinacci — Thu, 10 Mar 2016 18:15:00 GMT

Hi Nikolay,

I have the same problem: when multiple failures on main ring occur the ports on interconnected nodes facing sub-ring go to blocked state.

Have you done any advancement on this?

Thank you for the help.

RE: ERPS subring problem.

imhasanawan — Wed, 04 Aug 2021 22:58:00 GMT

hey everyone..

i am using ERPS for the first time.
and i am going from EAPS to ERPS..
so i have to make changes in my existing running network for that.

i have a scenario i am showing in a picture to elaborate the issue i am facing.
Can any body help me out with best practice for this scenario.?

***at the moment my main ring is using EAPS and it is working fine.

the switch4 has 2 primary from switch1 and secondary from switch2 but no auto failover at the moment
i manually shift it to secondary path if primary goes down.
also the reason for choosing ERPS is that i want shifting automation for switch4 .

.these sites are far from each other and i have shown the connectivity diagram which is at the moment. i cannot change the connection points on any of the switches.

Thank U
Hasan
Lahore, Pakistan.