topic New ARP entries is not learned on X480 in ExtremeSwitching (EXOS/Switch Engine)

New ARP entries is not learned on X480

Richard_Wenners — Tue, 18 Apr 2017 12:16:00 GMT

We had a strange issue yesterday. One of our Summit X480 (with VIM2-10G4X card) didn't learned new ARP entries. When I tried to pinged the IPs (which has not learned) from the X480 I got error message "Packet transmit error" . A few houres earlier a got the following message in the logfile: " IPv4 multicast entry not added. Hardware L3 Table full."

After a reboot, it started to work again.

show iparp
Dynamic Entries : 7655 Static Entries : 0
Pending Entries : 10
In Request : 4631721337 In Response : 175743045
Out Request : 15168044352 Out Response : 4475702318
Failed Requests : 2508625354
Proxy Answered : 4079619495
Rx Error : 243652 Dup IP Addr : 212.37.109.1
Rejected Count : 25638473 Rejected IP : 212.112.190.157
Rejected Port : 25 Rejected I/F : VL_IN_IP-ONLY_M

Max ARP entries : 16384 Max ARP pending entries : 256
ARP address check: Enabled ARP refresh : Enabled
Timeout : 20 minutes ARP Sender-Mac Learning : Disabled
Locktime : 1000 milliseconds
Retransmit Time : 1000 milliseconds
Reachable Time : 900000 milliseconds (Auto)
Fast Convergence : Off

show iproute reserved-entries statistics
|-----In HW Route Table-----| |--In HW L3 Hash Table--|
# Used Routes # IPv4 Hosts IPv4 IPv4 IPv6 IPv4
Slot Type IPv4 IPv6 Local Remote Local Rem. Local MCast
---- --------------- ------- ------ ------ ------ ----- ----- ----- -----
1 X480-24x(10G4X) 414 0 7654 0 0 0 0 18

Theoretical maximum for each resource type:
X440 32 16 64 64 509 512 256 * 256
E4G-200 12256 6128 8189 12288 8189 8192 4096 * 6000
X460, E4G-400 12256 6128 12288 12288 16381 16384 8192 * 6000
X480(40G4X) 16352 8176 8189 16384 8189 8192 4096 * 4096
X670-48x, X670V-48x 16352 8176 8189 16384 8189 8192 4096 * 4096
X670V-48t 16352 8176 16381 16384 16381 16384 8192 * 6000
X770 16352 8176 16384 16384 131072 49152 49149 *106496
X480 262112 8192 16381 40960 16381 16384 8192 * 6000

What is the L3 host limit on a X480 with VIM2-10G4X card? Is it 8000 or 16000 host entries?

EXOS 15.5.3.4 patch1-5

RE: New ARP entries is not learned on X480

Stephane_Grosj1 — Tue, 18 Apr 2017 13:14:00 GMT

Hi,

it's 16K.
What is happening?

RE: New ARP entries is not learned on X480

Richard_Wenners — Tue, 18 Apr 2017 14:11:00 GMT

Hi

After the reboot everything seems to works. If the limit is 16000 L3 hosts the number of ARPs isn't the problem. We have around 8000 ARP entries. I have not change the "External lookup tables" (default L2-and-L3). Are you sure that I not have to change this to "L3-only" to increase limit to 16000 entires?

I found this post https://gtacknowledge.extremenetworks.com/articles/How_To/Multicast-Entry-not-Added-Hardware-Table-F...

So I have now tried to disable IGMP snooping, maybe there was a rogue client who sent a lot of multicast streams.

The problem is that the switch still snoops multicast groups!

disable igmp snooping vlan "VL_SN_VPLS_2000"

clear igmp snooping "VL_SN_VPLS_2000"

show igmp snooping cache vlan VL_SN_VPLS_2000
Snooping/MVR Cache Timeout: 300 sec

Type Group Sender Age InVlan

snoop 224.0.0.9 92.244.217.6 22 VL_SN_VPLS_2000
Vlan Port Vid
VL_SN_VPLS_2000 29 2000

snoop 239.192.152.143 92.244.216.46 261 VL_SN_VPLS_2000
Vlan Port Vid
VL_SN_VPLS_2000 29 2000

snoop 239.192.152.143 92.244.218.238 35 VL_SN_VPLS_2000
Vlan Port Vid
VL_SN_VPLS_2000 29 2000

Multicast cache distribution:
189 entries from Snooping 0 entries from MVR 0 entries from PIM

VL_SN_VPLS_2000: Multicast cache distribution:
61 entries from Snooping 0 entries from MVR 0 entries from PIM

Total Cache Entries: 189
Total Cache Entries for VLAN VL_SN_VPLS_2000: 61

Do I have to reboot the switch for these changes to take affect?

RE: New ARP entries is not learned on X480

Stephane_Grosj1 — Tue, 18 Apr 2017 17:02:00 GMT

Sorry, I didn't see the problem statement at first.
The message was pointing to a fully used HW table (L3 Hash), is the above an output at the time of the issue? Because there's no Multicast usage in it...
Maybe it's more of a bug?

RE: New ARP entries is not learned on X480

Richard_Wenners — Tue, 18 Apr 2017 17:15:00 GMT

The output was made when we had the ARP issue. Not when the switch logged message "IPv4 multicast entry not added. Hardware L3 Table full". This message happen a few hours earlier.

Do you recommed to upgrade EXOS?

What about IGMP snooping. Why does the switch making a mcast cache when igmp snooping is disabled? Tried to disable igmp snooping on a X620 same result (the switch still making a mcast cache).

RE: New ARP entries is not learned on X480

Stephane_Grosj1 — Tue, 18 Apr 2017 17:50:00 GMT

The HW has room for 16K ARP, but it's sharing space with Multicast. Multicast has priority over ARP. So, if you were filling up the HW with Multicast, that could impact ARP.

For ARP, you also have to make sure the iparp cache is large enough (but I saw in your output that it's set to 16K as well, so that's good).

Do you have a lot of multicast, so that it could fill the 6000 entries in HW? As this is a hash table, collision happens way before.

Not sure about the disable igmp snooping.

RE: New ARP entries is not learned on X480

Richard_Wenners — Tue, 18 Apr 2017 18:46:00 GMT

We are using the X480 switch to route Internet traffic for our customers (we are a small ISP). So we do not have any multicast needs. But some customer maybe have a misconfigured devices and sending multicast stream to our router (X480). Maybe we shall apply an ACL to block multicast traffic from the customers to protect the HW resources on the X480?

What does "ARP total" means when you this command?

# show iparp stats summary
IP ARP VR Statistics Tue Apr 18 15:44:18 2017
ARP-
ARP Total Dynamic Static Pending Unneeded Failed (Rejected)
===============================================================================
Totals for all VRs
12145 7807 0 255 139 3897236 11871
===============================================================================

RE: New ARP entries is not learned on X480

Erik_Auerswald — Tue, 18 Apr 2017 18:46:00 GMT

The Total entries counter reflects the total number of entries that are currently allocated and not freed. Hence they also include Failed entries as well as ARP unneeded entries.

RE: New ARP entries is not learned on X480

Stephane_Grosj1 — Tue, 18 Apr 2017 18:46:00 GMT

This output is from the software side, not the HW utilization. You need to configure the software side to be greater than your expected usage. To know the HW usage, that's the sh iproute reserved-entries stat command, that you know already.