New ARP entries is not learned on X480

  • 0
  • 1
  • Question
  • Updated 1 year ago
  • Answered

We had a strange issue yesterday. One of our Summit X480 (with VIM2-10G4X card) didn't learned new ARP entries. When I tried to pinged the IPs (which has not learned) from the X480 I got error message "Packet transmit error" . A few houres earlier a got the following message in the logfile: "<Warn:Kern.IPv4Mc.Warning> IPv4 multicast entry not added.  Hardware L3 Table full."

After a reboot, it started to work again.


show iparp
Dynamic Entries  :        7655             Static Entries            :          0
Pending Entries  :          10
In Request       :  4631721337             In Response               :  175743045
Out Request      :  15168044352             Out Response              : 4475702318
Failed Requests  :  2508625354
Proxy Answered   :  4079619495
Rx Error         :      243652             Dup IP Addr               :    212.37.109.1
Rejected Count   :    25638473             Rejected IP               : 212.112.190.157
Rejected Port    :          25             Rejected I/F              : VL_IN_IP-ONLY_M

Max ARP entries  :       16384             Max ARP pending entries   :        256
ARP address check:    Enabled              ARP refresh               :    Enabled
Timeout          :          20 minutes     ARP Sender-Mac Learning   :   Disabled
Locktime         :        1000 milliseconds
Retransmit Time  :        1000 milliseconds
Reachable Time   :      900000 milliseconds (Auto)
Fast Convergence :         Off


show iproute reserved-entries statistics
                      |-----In HW Route Table-----|   |--In HW L3 Hash Table--|
                       # Used Routes   # IPv4 Hosts   IPv4   IPv4  IPv6   IPv4
Slot  Type              IPv4   IPv6    Local Remote   Local  Rem.  Local  MCast
----  --------------- ------- ------  ------ ------   -----  ----- -----  -----
1     X480-24x(10G4X)     414      0    7654      0       0      0     0     18

Theoretical maximum for each resource type:
X440                       32     16      64     64     509    512   256 *  256
E4G-200                 12256   6128    8189  12288    8189   8192  4096 * 6000
X460, E4G-400           12256   6128   12288  12288   16381  16384  8192 * 6000
X480(40G4X)             16352   8176    8189  16384    8189   8192  4096 * 4096
X670-48x, X670V-48x     16352   8176    8189  16384    8189   8192  4096 * 4096
X670V-48t               16352   8176   16381  16384   16381  16384  8192 * 6000
X770                    16352   8176   16384  16384  131072  49152 49149 *106496
X480                   262112   8192   16381  40960   16381  16384  8192 * 6000


What is the L3 host limit on a X480 with VIM2-10G4X card? Is it 8000 or 16000 host entries?

EXOS 15.5.3.4 patch1-5

Photo of Richard Wennerstrom

Richard Wennerstrom

  • 140 Points 100 badge 2x thumb

Posted 1 year ago

  • 0
  • 1
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 12,552 Points 10k badge 2x thumb
Hi,

it's 16K.
What is happening?
Photo of Richard Wennerstrom

Richard Wennerstrom

  • 140 Points 100 badge 2x thumb

Hi

After the reboot everything seems to works. If the limit is 16000 L3 hosts the number of ARPs isn't the problem. We have around 8000 ARP entries. I have not change the "External lookup tables" (default L2-and-L3). Are you sure that I not have to change this to "L3-only" to increase limit to 16000 entires?

I found this post https://gtacknowledge.extremenetworks.com/articles/How_To/Multicast-Entry-not-Added-Hardware-Table-F...

So I have now tried to disable IGMP snooping, maybe there was a rogue client who sent a lot of multicast streams.

The problem is that the switch still snoops multicast groups!

disable igmp snooping vlan "VL_SN_VPLS_2000"

clear igmp snooping "VL_SN_VPLS_2000"

show igmp snooping cache vlan VL_SN_VPLS_2000
Snooping/MVR Cache Timeout: 300 sec

 Type Group             Sender               Age  InVlan

snoop 224.0.0.9         92.244.217.6         22   VL_SN_VPLS_2000
      Vlan             Port        Vid
      VL_SN_VPLS_2000  29          2000

snoop 239.192.152.143   92.244.216.46        261  VL_SN_VPLS_2000
      Vlan             Port        Vid
      VL_SN_VPLS_2000  29          2000

snoop 239.192.152.143   92.244.218.238       35   VL_SN_VPLS_2000
      Vlan             Port        Vid
      VL_SN_VPLS_2000  29          2000

Multicast cache distribution:
   189 entries from Snooping           0 entries from MVR           0 entries from PIM

VL_SN_VPLS_2000: Multicast cache distribution:
    61 entries from Snooping           0 entries from MVR           0 entries from PIM


Total Cache Entries: 189
Total Cache Entries for VLAN VL_SN_VPLS_2000: 61


Do I have to reboot the switch for these changes to take affect?

Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 12,552 Points 10k badge 2x thumb
Sorry, I didn't see the problem statement at first.
The message was pointing to a fully used HW table (L3 Hash), is the above an output at the time of the issue? Because there's no Multicast usage in it...
Maybe it's more of a bug?
Photo of Richard Wennerstrom

Richard Wennerstrom

  • 140 Points 100 badge 2x thumb

The output was made when we had the ARP issue. Not when the switch logged message "IPv4 multicast entry not added. Hardware L3 Table full". This message happen a few hours earlier.

Do you recommed to upgrade EXOS?

What about IGMP snooping. Why does the switch making a mcast cache when igmp snooping is disabled? Tried to disable igmp snooping on a X620 same result (the switch still making a mcast cache).

Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 12,552 Points 10k badge 2x thumb
The HW has room for 16K ARP, but it's sharing space with Multicast. Multicast has priority over ARP. So, if you were filling up the HW with Multicast, that could impact ARP.

For ARP, you also have to make sure the iparp cache is large enough (but I saw in your output that it's set to 16K as well, so that's good).

Do you have a lot of multicast, so that it could fill the 6000 entries in HW? As this is a hash table, collision happens way before.

Not sure about the disable igmp snooping.
Photo of Richard Wennerstrom

Richard Wennerstrom

  • 140 Points 100 badge 2x thumb

We are using the X480 switch to route Internet traffic for our customers (we are a small ISP). So we do not have any multicast needs. But some customer maybe have a misconfigured devices and sending multicast stream to our router (X480). Maybe we shall apply an ACL to block multicast traffic from the customers to protect the HW resources on the X480?

What does "ARP total" means when you this command?


# show iparp stats summary
IP ARP VR Statistics                                   Tue Apr 18 15:44:18 2017
                                                ARP-
   ARP Total    Dynamic     Static    Pending   Unneeded     Failed  (Rejected)
===============================================================================
Totals for all VRs
       12145       7807          0        255        139    3897236       11871
===============================================================================

Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 12,782 Points 10k badge 2x thumb
The Total entries counter reflects the total number of entries that are currently allocated and not freed. Hence they also include Failed entries as well as ARP unneeded entries.
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 12,552 Points 10k badge 2x thumb
This output is from the software side, not the HW utilization. You need to configure the software side to be greater than your expected usage. To know the HW usage, that's the sh iproute reserved-entries stat command, that you know already.