topic RE: Don't see macs in fdb table on x450a-24x in ExtremeSwitching (EXOS/Switch Engine)

Don't see macs in fdb table on x450a-24x

Victor_Vit — Wed, 10 Jan 2018 18:29:00 GMT

Hello.
There is a problem with adding some random client's macs in fdb table on extreme x450a-24x firmware version 15.3.5.2.
I see this macs on previous switches, clients get ip by DHCP, but when I do
show fdb 11:11:11:11:11:11 - I don't see this mac (for exemple) in fdb table. Extreme switch works also as gateway for clients and iparp record is absent.
BUT, when I do
clear fdb vlan VlanClient1
and after
show fdb 11:11:11:11:11:11
I see this mac on switch. Also appears record in iparp table.
Number of all macs is near 4200-4300

Any ideas what is it?

RE: Don't see macs in fdb table on x450a-24x

Jarek — Wed, 10 Jan 2018 19:31:00 GMT

Hi,

could you show us:

sh fdb stats
sh iproute reserved-entries statistics
debug hal show forwarding distributions system

--
Jarek

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Wed, 10 Jan 2018 19:31:00 GMT

pgs.20 # sh fdb stats

Total: 4318 Static: 0 Perm: 0 Dyn: 4318 Dropped: 0
FDB Aging time: 300

pgs.21 # sh iproute reserved-entries statistics
|-----In HW Route Table-----| |-------In HW L3 Hash Table------|
# Used Routes # IPv4 Hosts IPv4 IPv4 IPv6 IPv4 IPv6
Slot Type IPv4 IPv6 Local Remote Local Rem. Local MCast MCast
---- --------------- ------- ------ ------ ------ ----- ----- ----- ------ ------
1 X450a-24x 228 0 2922 0 0 0 0 207 0

pgs.22 # debug hal show forwarding distributions system
L3 Hash Table Utilization:
Used/Available: 212/8192

L3 Hash Table Bucket Distribution:
Total: 1024 buckets
[ 0 entries]: 835 buckets
[ 1 entry]: 167 buckets
[ 2 entries]: 21 buckets
[ 3 entries]: 1 bucket
[ 4 entries]: 0 buckets
[ 5 entries]: 0 buckets
[ 6 entries]: 0 buckets
[ 7 entries]: 0 buckets
[ 8 entries]: 0 buckets

RE: Don't see macs in fdb table on x450a-24x

Jarek — Wed, 10 Jan 2018 19:31:00 GMT

Probably you have reached supported limits on the device with default configuaration.

Release notes says:
(that are maximum values)
- FDB (maximum L2 entries- 16,384 (d)
- IPv4 ARP entries in hardware with maximum LPM routes - 2000 (f)
- IPv4 remote hosts in hardware with zero LPM routes - 14000 (f)

(d) - Effective capacity varies based on actual MAC addresses and VLAN IDs used and hash algorithm selected
(f) - Effective capacity varies based on actual IP addresses and hash algorithm selected

All this features are using hardware and depends on actual utilization + reserved space + configuration.

Default LPM tables in X450a is set to 12240.
I see that now you have 228 LPM entries.

If you don't need to much space for routes you can set it to lower value,
for example: configure iproute reserved-entries 2000

See also info bellow:

configure iproute reserved-entries
default IPv4 hosts can occupy at least 16 entries in HW route table
maximum Reserve entire hardware table for IPv4/IPv6 routes
Sum of IPv4 routes plus twice number of IPv6 routes to reserve. IPv4 hosts can occupy unused space.

From EXOS_Command_Reference_Guide_15_3.pdf

======================================================
Demand on the Layer3 Hash table can be reduced by allowing IPv4 hosts to be stored in the LPM tables instead.
This command allows you to reserve a portion of the LPM tables for routes,
and this creates an unreserved portion that can be used to store IPv4 hosts.
For more information, see the Extended IPv4 Host Cache section in the Extreme XOS Concepts Guide.

The default setting can support most networks, but if more than a few hundred local IP hosts and IP multicast entries are present, you can improve switch performance by calculating and configuring the reserved space for route entries to allow unreserved space for IPv4 hosts.
Changing the number of reserved route entries does not require a reboot of the affected slots or switches.

If the switch contains fewer routes than the capacity of the LPM tables,
the number of route entries toreserve for a slot or switch should be the number of routes currently used in the hardware tables, plus an additional cushion for anticipated growth. Because each IPv6 route takes up the space of two IPv4 routes, the number of route entries to reserve is two times the value in the IPv6 routes column, plus the value in the IPv4 routes column, plus room for anticipated growth.
For example, if you want to reserve space for 100 IPv4 routes and 20 IPv6 routes, the required number of route entries is 140 (100 + 2*20).

======================================================

You can also check the EXOS_Concepts_Guide_15_3.pdf

--
Jarek

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Wed, 10 Jan 2018 19:31:00 GMT

I have done this some month ago.
See:
pgs.23 # sh iproute reserved-entries
IPv4 # Reserved Routes Minimum #
Slot Type Routes IPv4 (or IPv6) IPv4 Hosts
---- ---------------- -------- ------ ------------------ ----------
1 X450a-24x Internal 1000 ( 500) 11256

pgs.24 # sh conf | i reser
configure iproute reserved-entries 1000

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Fri, 12 Jan 2018 00:24:00 GMT

OK, when I
enable log debug-mode
and made filter
configure log filter "filt1" add events "HAL.FDB.DebugSummary"

I get next messages in log for ones of this macs with problem:

Jan 11 18:28:18 x.x.x.x Jan 11 18:28:17 HAL.FDB: Add Entry [cardId = ffffffff] vlanInst: 1000035 mac: XX:XX:XX:XX:XX:XX port 00010006, flags 00000021
Jan 11 18:28:18 x.x.x.x Jan 11 18:28:17 HAL.FDB: Could not find a free entry
Jan 11 18:28:18 x.x.x.x Jan 11 18:28:17 HAL.FDB: Add IpAdj entry: vrId 00000002 ipAddr 01010101 vlanInstance 000f42d1 Mac XX:XX:XX:XX:XX:XX slotPort 00010006
Jan 11 18:28:18 x.x.x.x Jan 11 18:28:17 HAL.FDB: Del IpAdj entry: vrId 00000002 ipAddr 01010101 vlanInstance 000f42d1

Can anyone explains why It can not find free entry and how can I check this entry?

RE: Don't see macs in fdb table on x450a-24x

Jarek — Fri, 12 Jan 2018 17:13:00 GMT

What firmware version do you have ? 15.3.5.2-patch-??

Please see also this bellow (How does a hash collision happen😞
https://gtacknowledge.extremenetworks.com/articles/Solution/Warn-HAL-FDB-L2SlotTblFull-MSM-A-FDB-ent...

--
Jarek

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Fri, 12 Jan 2018 17:13:00 GMT

ExtremeXOS version 15.3.5.2 v1535b2-patch1-10
We did this recommendation last night and now we are monitoring and waiting on result.

On other switch we found 2 macs, that can not work together. If deleted first - second works fine but after that - this first can not add to all tables and also when deleted second - first works but after that - this second can not add to all tables.

How I understand - there is a problem in extreme's algorithm with some macs in some situation...

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Fri, 12 Jan 2018 17:13:00 GMT

crc16 didn't help.

RE: Don't see macs in fdb table on x450a-24x

Jarek — Fri, 12 Jan 2018 17:13:00 GMT

I know that this is not a "solution", but:

1) did you try to add those MACs as a static ?

2) I see also that last stable EXOS is 15.3.5.2-patch1-14 (latest is p1-17) - maybe upgrade will fix your issue - please see the release notes

And question to Extreme Guys - Is it possible to improve hashing algo in next firmware release for 250e / X450a+e switches ?

--
Jarek

RE: Don't see macs in fdb table on x450a-24x

Victor_Vit — Fri, 12 Jan 2018 17:13:00 GMT

I can't add those MACs, I get error:
pgs # create fdb f4:f2:6d:83:25:c9 vlan Vlan-1414 ports 3
Error: Unable to create static entry for MAC f4:f2:6d:83:25:c9

But if I add another mac for this vlan and port - all is good :
pgs # create fdb f4:f2:6d:83:25:10 vlan Vlan-1414 ports 3
* pgs #
I get no errors

In release note for patch1-14 I found this text:
"MAC Address Hash
The hash algorithm used for the L2 MAC hash table has been improved so that more MAC addresses can be inserted prior to filling the hash bucket. This feature applies only to the Summit X670, BlackDiamond 8900- 40G6Xc, and BlackDiamond X8 series switches."

Our model x450a-24x is not in the list. And We don't now or it is there any reason to
upgrade...