https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39424#M8683 Hello Jarek,<BR /> Yes, we sent the ping to directly to customers.<BR /> No High cpu was found on Juniper.<BR /> <BR /> Thats right, when add the ports 45 and 46 I see the packet loss.<BR /> Yes I tried all of them, even fixed and customs. Fri, 15 Jan 2016 20:38:00 GMT Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39413#M8672 Hello,<BR /> <BR /> There's something odd happenings here, and more than that was the solution of it.<BR /> <BR /> We've a sharing with a Juniper MX80, and there're two 10G ports connect on, the 47 and 48 (Extreme).<BR /> After issuing the command:<BR /> <BR /> configure sharing 47 add ports 45,46<BR /> <BR /> The hashings seens to not work properly.<BR /> The traffic droped significantly, the two added ports (45,46) seems to be more affected than the 47,48.<BR /> <BR /> Then comes the odd "solution", after disabling the sharing and adding again with the master becoming port 45, the problem was solved.<BR /> <BR /> It's a problem to be checked, because it's the second time that I'm having the same problem.<BR /> I'm using the version 15.6.3.1 v1563b1-patch1-5.<BR /> X670. Fri, 15 Jan 2016 18:28:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39413#M8672 Julian_Eble 2016-01-15T18:28:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39414#M8673 Hi,<BR /> <BR /> you mean traffic from X670 to MX80 OR from MX80 to X670 ?<BR /> What hash algo. do you use ?<BR /> <BR /> --<BR /> Jarek<BR /> Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39414#M8673 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39415#M8674 From MX to X670.<BR /> I'm using custom and I did tried all of them.<BR /> I know it's strange because the hash is due to TX traffic.<BR /> We tried to do some changes in Juniper but nothing could solve this.<BR /> Only the disable/enable sharing solved. Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39415#M8674 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39416#M8675 Juniper MX 80 has a Trio chipset and can only balance per-flow not per packet.<BR /> If you have high traffic flow on one port it will stay on that port. Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39416#M8675 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39417#M8676 Well but why, after disabling and enabling the extreme sharing the traffic flows well?<BR /> And packet loss stoped Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39417#M8676 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39418#M8677 I think the flows (active connections) are dropped because of sharing reconfiguration and all new are well balanced. Have you try add a port and wait about 10-15 min,<BR /> and then check the ports utilization? Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39418#M8677 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39419#M8678 Where do you observe packet loss ? On X670 or Juniper ?<BR /> Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39419#M8678 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39420#M8679 It's hard to say,the counters are not showing the drop. But BGP sessions are dropped and some packet loss (ping) are noted. Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39420#M8679 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39421#M8680 BGP session probably are dropped because of of sharing reconfiguration.<BR /> <BR /> Have you try add a port and wait about 10-15 min,<BR /> and then check the ports utilization?<BR /> <BR /> When do you observe packet loss ? <BR /> After you add a new port or after change the master port for LAG ?<BR /> --<BR /> Jarek<BR /> <BR /> Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39421#M8680 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39422#M8681 Hello Jarek,<BR /> We tried for 5 minutes, we couldn't perform more than that.<BR /> I observe after adding the two ports to the LAG.<BR /> <BR /> Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39422#M8681 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39423#M8682 You send ping to Juniper interface ? <BR /> If yes, have you try to ping something that is after the Juniper ?<BR /> Maybe cpu on Juniper is busy...<BR /> <BR /> If I understand correctly, you have two port in LAG 47 and 48. 47 is master port.<BR /> When you add 45 and 46 to this LAG, you see problems with traffic.<BR /> Do you change the hash algorithm ?<BR /> Do you use L3_L4 hash or custom hash ?I ask because bellow you have pasted show's with two of them.<BR /> <BR /> --<BR /> Jarek<BR /> Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39423#M8682 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39424#M8683 Hello Jarek,<BR /> Yes, we sent the ping to directly to customers.<BR /> No High cpu was found on Juniper.<BR /> <BR /> Thats right, when add the ports 45 and 46 I see the packet loss.<BR /> Yes I tried all of them, even fixed and customs. Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39424#M8683 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39425#M8684 Any info in logs on X670 ?<BR /> Did you check congestion counter on ports?<BR /> What shows: debug hal show congestion (hit a few times)<BR /> <BR /> Did you check CPU utilization on X670 after you add a port to the LAG?<BR /> <BR /> --<BR /> Jarek Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39425#M8684 Jarek 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39426#M8685 No infos.<BR /> Yes, I did check that, but nothing incremented.<BR /> I don't have the problem now, because after re-creating the lag it stoped.<BR /> No, I haven't see CPU usage. Fri, 15 Jan 2016 20:38:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39426#M8685 Julian_Eble 2016-01-15T20:38:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39427#M8686 Can you send the "show sharing" output and explain how you are determining the traffic is being affected? Fri, 15 Jan 2016 21:14:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39427#M8686 Patrick_Voss 2016-01-15T21:14:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39428#M8687 Load Sharing Monitor<BR /> Config Current Agg Ld Share Ld Share Agg Link Link Up<BR /> Master Master Control Algorithm Group Mbr State Transitions<BR /> ==============================================================================<BR /> 8 8 LACP L3_L4 8 Y A 3<BR /> L3_L4 9 Y A 3<BR /> 27 27 LACP L3_L4 27 Y A 1<BR /> L3_L4 28 Y A 1<BR /> 29 29 LACP L3_L4 29 Y A 1<BR /> L3_L4 30 Y A 1<BR /> L3_L4 31 Y A 1<BR /> 47 45 LACP L3_L4 45 Y A 1<BR /> L3_L4 46 - R 0<BR /> L3_L4 47 Y A 2<BR /> L3_L4 48 Y A 2<BR /> ==============================================================================<BR /> <BR /> Below I enabled just one port to see if the problems happens.<BR /> After that the traffic became this:<BR /> <BR /> Port Link Link Rx Peak Rx Tx Peak Tx State Speed % bandwidth % bandwidth % bandwidth % bandwidth<BR /> ================================================================================<BR /> 45 A 10000 1.59 2.96 0.99 1.09<BR /> 46 R 0 0.00 0.00 0.00 0.00<BR /> 47 A 10000 26.33 38.23 34.27 50.37<BR /> 48 A 10000 24.27 40.51 43.32 63.29<BR /> <BR /> Port 45 has a much less traffic then the others ports.<BR /> We have about 5 lacp on this switch with the Juniper, none had this kind of problem.<BR /> <BR /> The only behaviour was about using predecessors ports of the master sharing port.<BR /> As I said before, after disabling/enabling the sharing and set the port 45 as Master the traffic became ok.<BR /> <BR /> Here's the output now:<BR /> <BR /> X670 # sh sharingLoad Sharing Monitor<BR /> Config Current Agg Ld Share Ld Share Agg Link Link Up<BR /> Master Master Control Algorithm Group Mbr State Transitions<BR /> ==============================================================================<BR /> 8 8 LACP L3_L4 8 Y A 4<BR /> L3_L4 9 Y A 3<BR /> 27 27 LACP L3_L4 27 Y A 1<BR /> L3_L4 28 Y A 1<BR /> 29 29 LACP L3_L4 29 Y A 1<BR /> L3_L4 30 Y A 1<BR /> L3_L4 31 Y A 1<BR /> 45 45 LACP custom 45 Y A 2<BR /> custom 46 Y A 2<BR /> custom 47 Y A 2<BR /> custom 48 Y A 2<BR /> ==============================================================================<BR /> Link State: A-Active, D-Disabled, R-Ready, NP-Port not present, L-Loopback<BR /> Load Sharing Algorithm: (L2) Layer 2 address based, (L3) Layer 3 address based<BR /> (L3_L4) Layer 3 address and Layer 4 port based<BR /> (custom) User-selected address-based configuration<BR /> Custom Algorithm Configuration: ipv4 L3-and-L4, crc-32 upper<BR /> Number of load sharing trunks: 4<BR /> <BR /> X670 # sh ports 45-48 utilization bandwidth<BR /> Port Link Link Rx Peak Rx Tx Peak Tx<BR /> State Speed % bandwidth % bandwidth % bandwidth % bandwidth<BR /> ================================================================================<BR /> 45 A 10000 15.95 15.95 44.13 44.13<BR /> 46 A 10000 15.75 15.75 38.12 38.12<BR /> 47 A 10000 15.87 38.23 4.10 51.62<BR /> 48 A 10000 15.99 40.51 5.93 63.29<BR /> ================================================================================<BR /> &gt; indicates Port Display Name truncated past 8 characters<BR /> Link State: A-Active, R-Ready, NP-Port Not Present, L-Loopback Fri, 15 Jan 2016 21:14:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39428#M8687 Julian_Eble 2016-01-15T21:14:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39429#M8688 Based on this, it looks like what Jarek said is probably what happened. When the sharing was disabled on the X670, the LACP would drop causing the ports to be removed from the LAG on the MX80 as well.<BR /> <BR /> When it came back up, the existing flows from the MX80 were re-hashed including the new ports.<BR /> <BR /> -Brandon Fri, 15 Jan 2016 21:14:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39429#M8688 BrandonC 2016-01-15T21:14:00Z https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39430#M8689 I agree with that, but why when I do the same thing with other switchs this problem doesn't occur?<BR /> I mean, the thing is the master port number, if you add some ports that's not the sequential the same behavior appears.<BR /> I tested that with another switch and another MX, the MX had other firmware.<BR /> But I had the same symptoms Fri, 15 Jan 2016 21:14:00 GMT https://community.extremenetworks.com/t5/extremeswitching-exos-switch/problems-with-port-sharings-x670/m-p/39430#M8689 Julian_Eble 2016-01-15T21:14:00Z