topic RE: Rate-limit in ExtremeSwitching (EXOS/Switch Engine)

Rate-limit

eyeV — Thu, 20 Nov 2014 18:01:00 GMT

Hi everybody.
I have two Summit x670 (15.4.1.3) switches and I'd like to limit inbound broadcast, multicast and unknown unicast packets on specific ports. So, I've configured rate-limit to 500pps.

config port 3 rate-limit flood broadcast 500
config port 3 rate-limit flood multicast 500
config port 3 rate-limit flood unknown-destmac 500Then I see the output of "show ports 3 stat" command. I see only 10-20 pps, but Flood Rate Exceeded counter is increasing and I have log messages like
Flood Rate Limiting activated on Port 3

RE: Rate-limit

PARTHIBAN_CHINN — Thu, 20 Nov 2014 18:16:00 GMT

can you write an acl to count the ingress packets.
just to confirm what packets are seen.
show l2stats for that specific vlan.
see how many packets are multicast and how many are broadcast.
clear l2stats
show l2stats vlan

RE: Rate-limit

eyeV — Thu, 20 Nov 2014 18:32:00 GMT

I can write an acl like.
entry BCAST {
if {
ethernet-destination-address ff:ff:ff:ff:ff:ff;
}
then {
packet-count bcast-pkt;
}
}As to "show l2stats". I don't see any broadcast or multicast counters in output of this command. I see only
Bridge interface on VLAN Default:
Total number of packets to CPU = 2923.
Total number of packets learned = 38837.
Total number of IGMP control packets snooped = 255.
Total number of IGMP data packets switched = 218.
Total number of MLD control packets snooped = 0.
Total number of MLD data packets switched = 0.

RE: Rate-limit

PARTHIBAN_CHINN — Thu, 20 Nov 2014 18:39:00 GMT

hello broadcast packets directly hit cpu.
unknown unicast [mac learning]
Total number of packets learned = 38837.
clear l2stats
show l2stats vlan -run this command 5 times with 1 sec interval to see how many
packets are hitting cpu.

RE: Rate-limit

eyeV — Thu, 20 Nov 2014 18:51:00 GMT

Ok. But I have about 50 vlans on this port.

Anyway, this is an output of show l2stats vlan Default command.

Bridge interface on VLAN Default:
Total number of packets to CPU = 5.
Total number of packets learned = 48.

Bridge interface on VLAN Default:
Total number of packets to CPU = 8.
Total number of packets learned = 70.

Bridge interface on VLAN Default:
Total number of packets to CPU = 11.
Total number of packets learned = 104.

Bridge interface on VLAN Default:
Total number of packets to CPU = 14.
Total number of packets learned = 132.

Bridge interface on VLAN Default:
Total number of packets to CPU = 16.
Total number of packets learned = 142.

Bridge interface on VLAN Default:
Total number of packets to CPU = 18.
Total number of packets learned = 174.

RE: Rate-limit

eyeV — Thu, 20 Nov 2014 19:00:00 GMT

This is the counter from acl

Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 43

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 43

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 50

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 62

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 69

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 70

* xCore1.49 # sh access-list counter
Policy Name Vlan Name Port Direction
Counter Name Packet Count Byte Count
==================================================================
bcast-counter * 2 ingress
bcast-pkt 83

I've run this command with 1 sec interval.

RE: Rate-limit

PARTHIBAN_CHINN — Thu, 20 Nov 2014 19:54:00 GMT

These outputs doesnt justify there is an issue.
Next steps would be to take a packet capture on the ingress.
You can do it by yourself by port mirroring or.
if you reachout to TAC they can do TCPdump ---which needs debug password to get into debug mode.

could you also check the cpu utilisation.
is any specific process is high ex.bcmrx?

RE: Rate-limit

eyeV — Thu, 20 Nov 2014 20:16:00 GMT

Thanks. CPU utilization in normal state. I'm going to capture traffic on this interface.

Could it be the bug in XOS or something? I found this topic here https://community.extremenetworks.com/extreme/topics/problem_with_rate_limit_on_summit_x650-l8ftu

RE: Rate-limit

PARTHIBAN_CHINN — Thu, 20 Nov 2014 20:27:00 GMT

It could be but I would always do the maximum troubleshooting i can do and then reach out to TAC for confirmation.

RE: Rate-limit

eyeV — Thu, 20 Nov 2014 20:37:00 GMT

Ok. Thanks.

RE: Rate-limit

eyeV — Wed, 04 Feb 2015 12:39:00 GMT

Hi everybody.
It might be interesting for somebody. I've got some explanations from TAC. My problem with rate-limit is peculiarity or feature of platform.

One second is divided into 15.625 microseconds intervals. The rate-limiting mechanism occurs when the platform receives lots of packets in one 15.625 microsecond interval.

For example. I've configured
conf ports 25 rate-limit flood broadcast 100000

Rate-limiting mechanism occurs when the box receives ~1.5 packet in 15.625 microsecond.

RE: Rate-limit

Stephane_Grosj1 — Wed, 04 Feb 2015 20:25:00 GMT

Hi,

if you want to stick to a pps measuring, apply a meter, otherwise yes, the newer chipsets are not working anymore at the /s sampling rate. This is the reason why you may hit the rate-limit while you don't have that much of traffic on a per second time basis.

As for tcpdump (I saw it mentioned in this thread), if you plan to use it keep in mind that you're sniffing in software, so it takes potentially a lot of resources, which may have a bad side effect. So take care with that. Port mirroring is happening in hardware, it's better to use it.

RE: Rate-limit

eyeV — Wed, 04 Feb 2015 20:25:00 GMT

Thanks, useful advice.

RE: Rate-limit

M_Nees — Thu, 23 Jun 2016 12:44:00 GMT

Hi,

currently i fight on the same line - means i use rate-limit (1000pps on a 1GB Link) and wonder why rate-limit exceeded and the counters are far away from this 1000pps threshold.

I use a X670V-G2. i use rate-limit to avoid loops because RSTP is to complex to setup in my environment (that it works with netlogin and is compatible to enterasys).

So i think because the new (G2 Switches) are measures based on the mentioned 15,6 micro-secs time slots (tokens) the configured CLI threshold in packets per second will never working reliable. Means the ASIC cannot measure the configured CLI limit in pps reliable.

I think there is currently (nearly) an ON - OFF Situation regarding rate-limits.If you turn it on, set it to the max value 262144 - because then max. 4 Packets per time slot (15,625 micro-secs) are allowed OR at least to a value 131072 then 2 Packets allowed.

This is step back / disadvantage compared to the G1 Switch Platform - normally we expect the Switch will offer better features!

Currently there is a KB Article which explain that:
"How to rate-limit implement on EXOS platforms?" KB2628

Are my understandings right ? If not please correct me.

Regards

RE: Rate-limit

Stephane_Grosj1 — Thu, 23 Jun 2016 12:54:00 GMT

Hi, I believe a fix is included in 21.1 for this feature. You might want to try if you have G2.

RE: Rate-limit

M_Nees — Thu, 23 Jun 2016 13:08:00 GMT

Hi Stephane,

i have a look at the current Release Notes 21.1.1 - and i found only these:

Configuration of Overhead-bytes in Calculating Rate-Limiting and Rate-Shaping

But this changes nothing regarding the lost or not reliable rate-limit threshold on G2 Switches.

Regards

RE: Rate-limit

Stephane_Grosj1 — Thu, 23 Jun 2016 13:08:00 GMT

Did you test with 21.1?

RE: Rate-limit

M_Nees — Thu, 23 Jun 2016 13:08:00 GMT

Hi Stephane,
i do not test because if nothing i written to the release notes engineering do not change the regarding code!

Do you think or know that anything would be changed ?

RE: Rate-limit

Stephane_Grosj1 — Thu, 23 Jun 2016 13:08:00 GMT

From what I recall, this code has been fixed in 21.1, I had a thread with engineering on that topic. I never tested it, so that would be worth checking in a lab first.

RE: Rate-limit

Erik_Auerswald — Thu, 23 Jun 2016 15:10:00 GMT

Just to document this, I have encountered the issue with basically unusable broadcast limiters on X670 switches with different EXOS versions (15.2, 15.3, 15.6). 