How to Diagnose and Troubleshoot Unicast Flooding

FAQ_User — Sat, 28 Jun 2014 00:05:00 GMT

Article ID: 5322

Goals
How to troubleshoot unicast flooding
Find mac address

Symptoms
Unicast flooding
Seeing half of conversation in sniffer trace

Cause
From the perspective of a passive sniffer, a non-mirrored switch port should be seen to transmit just broadcasts, multicasts, and unknowns. If sorting a sniffer Host Table analysis (in, for example, Sniffer Pro) by "In Packets", shows that any station that received packets was not seen to transmit them, or vice versa; you likely have a unicast flooding problem.

Though unicast flooding will not prevent delivery of network traffic, it can be a concern from both efficiency and security perspectives.

Unicast flooding on switches running in traditional mode is normally due to the lack of a Source Address Table (SAT) entry on the switch, for the destination MAC. With an unknown destination, the packet is flooded. The reasons for the lack of SATable entry, on the other hand, are not quite as clear-cut.

Possible flooding scenarios:

Loadsharing NICs or servers

UDP traffic

doesn't require acknowledgements, so knowledge of the destination MAC could time out during a long file transfer.

Multiple routers in a redundantly routed network

Using both a centrally based router serving as a default gateway,and a firewall router that also has direct access to the local stations' network; what can happen is that outgoing traffic will first hit the central router, which will map to the firewall router, and out of the network. The return trip will first go to the firewall router, which will then talk directly to the local station, bypassing the central router.
When using a standby router protocol such as VRRP or HSRP, even if there is no failover, there are not necessarily any guarantees (especially in a multipath, equal-cost scenario) as to how the traffic will be returned from the remote side of load-sharing routers. If the "wrong" path is chosen, we again have an asymmetric flow.

If in examining the layer 2 information of flooded frames, either source or destination MAC consistently references a router; or, in examining layer 3 you find that the source and destination networks consistently differ; or you see multiple router MACs bound to a single IP address; then asymmetrical routing is a good possibility.
Is there more than one layer 3 router with direct access to the layer 2 fabric in question? A simple test would be to examine primary and secondary IP network numbers for all router ports in the network.
IP Routers generally, by default, send out an ICMP Redirect frame to an originating station, if they know of a more preferable path than themselves. Do the routers have ICMP redirect enabled, and are they running a routing awareness protocol such as RIP? What are the statically configured routes? If an end station has a self-referencing default gateway, it will ignore ICMP redirects. In that event, it will also need a router running proxy ARP, in order to communicate out of the local network.

Asymmetrical VLAN flow / Port Overlapping

(cross-vlan, within a switch) using separate FIDs for each VLAN. Unicast flooding would be likely if the VLAN used in one direction of traffic flow differs from the VLAN used in the opposite direction of traffic flow.

Overflowing Source Address tables.

Most devices have a capacity for at least 4000 addresses.

STP Aging Time (

dot1dTpAgingTime

down from 300 seconds to 15 seconds, as a result of Spanning Tree Topology Change events.

Solution

Use Loadsharing/Teaming software that utilizes a Multicast (rather than Unicast) destination MAC, and scope that traffic to a VLAN or to a set of ports (5708).
Ensure that the firewall router goes back through the central router in order to access the local network.
Plug both routers into the same switch.
Take advantage of the fact that both routers and end stations have to keep an active ARP entry of the next hop. If you set the ARP timeout of the router (20 minutes by default, on the X-Pedition router) or affected end station (10 minutes by default, on UNIX machines) similar to the SAT timeout of the switches (usually 5 minutes by default), there will be sufficient knowledge of the whereabouts of end stations to prevent flooding.
Set up a UNIX
```
cron
```
job to periodically ping out of the "silent" station (using the "silent" MAC!) to a non-existent IP host on its subnet, thereby ARPing to all devices, making them aware of its location.
Set up a UNIX
```
cron
```
job to periodically ping the "silent" station.
Use a common FID - SVL mode - among VLANs that share traffic within a switch.

2839, "Flooding Unicast Traffic due to Load-sharing NICs"
4918, "In 802.1Q, what is the difference between IVL and SVL?
3460, "Changing Source Address Time-out on Switches and Bridges"
For a switch, use SPEL/EMAN's Device / Find Source Address to query for a MAC address. The SATable source port will blink.
For a switch, use SPEL/EMAN's Device / Bridge Status / Bridge / Filtering Databaseto view the SATable, and/or to configure a 'Static Filtering Database' entry.
For a SmartSwitch 2nd/3rd Gen running 5.01.33 firmware or higher, the
```
Network Tools
```
command '
```
show mac
```
' can be used to view the SAT, including FID identifier; plus it has sub-commands for granularity.
Use SPEL/EMAN's Device / System Group / Other Groups / Net to Media Table to view the ARP Cache, aka ipNetToMediaTable (this might help).
For a SmartSwitch, the
```
Network Tools
```
command '
```
show ip
```
' can be used to view the ARP Cache, aka ipNetToMediaTable, plus other misc data.

topic How to Diagnose and Troubleshoot Unicast Flooding in FAQs

How to Diagnose and Troubleshoot Unicast Flooding