IPV6 broadcast storm, on an IPV4 network


Userlevel 3
We have a number of printers that by default have IPV6 configured though not used. ( not supported by me )
These printers are also connected to an IPV4 vlan where the IP address is allocated via DHCP and routed at a distribution switch X460.

Recently we had a issue with only the printers on the network , other devices on the same network , were able to operate and use the dhcp service and access the internet.

Only these printers would not work , if we connected a laptop to the switch port or disconnected the printer and used that port , the laptop would operate as expected.

We are being informed by the printer manufacturer that the issue we are seeing is that the Printer is creating an IPV6 broadcast storm , which I could understand , though my question is :

Would this not present itself the same as an IPV4 broadcast storm , CPU BCMrx high , vlan unresponsive etc?

This might be a IPV6 multicast issue as well ..

I.E the IPV4 network would also be effected .. which it was not.

I look forward to any comments and or suggestions .

Regards

14 replies

Userlevel 6
Hi Rod,

How high is BCMRX getting in the "top" output? Is the issue still happening or is there a workaround to resolve it? If the issue is resolvable how long before it comes back?

There is a way to look at the packets hitting the CPU but this requires a case with GTAC and debug-mode to be enabled. Pending valid entitlement on the switch this may be the best route moving forward.
Userlevel 5
Hello Rod,

fwiw, I agree with your doubt about the diagnosis. Also agree with your reasoning - a traffic flood problem would leave tracks you would be able to follow.

On the flip side - no help understanding what's really going on.

Regards,

Mike
Userlevel 6
Hi Rod,

I took a second look at you post and misunderstood your question. High CPU will only appear if there is a lot learning going on. This traffic is automatically sent to the CPU for processing. If the traffic is being switched normally then you wouldn't see an issue unless the port is being over utilized. Without knowing what kind of traffic this is it would be very difficult to understand why this is only affecting the printers. Maybe you can mirror one of the printer ports and do a packet capture.
Userlevel 6
wonder if this is related to mcast listening floods we had back in 2014 when intel had bad drivers and if IPV6 were enabled and the nic card went to sleep.... ??
The issue is not a BROADCAST store. IPv6 does not use Broadcast, only multicast. Since we are not using IPv6 on the network yet I can't be of any more help than to eliminate what the vendor said so you aren't going down a wasted path. The first thing I ALWAY do when there is a problem like this is mirror the port and capture the data with Sniffer or Wireshark. Only then can you see what the traffic is and what it is really doing.
Userlevel 3
Hi many thanks for your comments , the engineers on site did take a trace and found that this is as suggested an IPV6 Multicast Listener Report ( STORM) .. so it would seem that the following scenario exists.

As we can have multiple networks on a single "piece of wire " ( old school ) we have the standard IPV4 behaving as expected and we also have the IPV6 ( Printer Default configuration ) sharing the same physical wire , though on a separate network , because all the printers are on the IPV6 network as well as the IPV4 , when one of them starts this IPV6 Query the others response and this seems to start a cyclic process that is "self destruction".

It has been suggested on other forums that Storm control , and other edge configurations could help , the main one being disabling IPV6 on the printers.

So I seem to have a fix ,Though I still do not understand how the IPV6 packets , hitting the Switch CPU , would not register as high BCMrx and then be dropped or were theses IPV6 packets hitting another process , and therefore we missed this .

Any advise on how IPV6 hits the cpu and the process used would be appreciated ..

Many Thanks again
Userlevel 6
Hi Rod,

How are we confirming that they are indeed hitting the CPU? It seems very odd to me that only the printers that are producing the traffic are experiencing issues even though there are multiple VLANs traversing the same physical interfaces. I do not think there is enough information to provide an explanation for what is happening on this network.
Userlevel 3
Hi, maybe you have arp storm or other traffic that is hitting the cpu? Wireshark is your friend as Derek suggested. You can also block ipv6 traffiic via vlan acl if you want. -- Jarek
Userlevel 3
we've had exactly the same issue.

as mentioned above it's not a broadcast storm (often caused when there's a loop in the network) but a lot of spurious IPV6 multicast packets being sent from one or two machines with a particular NIC (intel i217's from memory).

I think the switches and other non-v6 devices can cope perfectly well with the amount of traffic being forwarded, but the printers themselves get overloaded and stop functioning correctly, perhaps because they are the only devices listening to IPv6 multicasts?

if an actual broadcast/multicast storm was happening then I believe the switches would also start to suffer.
Userlevel 3
Hi

Thanks for the updates ..

We have a wireshark trace and have identified a number of "IPV6 top senders as previously suggested.

We are also going to control the IPv6 multicast by ACL.

As suggested in the posts , I think the Printer issues are only an " indicator" that there is an IPV6 multicast issue on the network, and we have seen no immediate effect on network performance.

Again thansk for all your comments and suggestions.,
Userlevel 5
Hello Rod,

Yours wont be the last time the community runs across this or related behavior.

Pain for your network will become a signpost in another troubleshoot. This forum is searchable from google - I'll also add a knowledge base doc.

First time I've personally heard about this sort of issue and isolated printer device-type behavior. On the other hand I learn networking on this forum daily.

Thank you for taking a minute to follow up.

Best regards,

Mike
Hello All,

Did you guys find solution for above problem as i am facing same issue on my network now. I captured packets from Wire shark software and found 5 desktop systems in my network who are broadcasting IPv6 packets in network. Right now i am disabling IPv6 TCP/IP option in those desktop systems lets see what will happen!!!
Userlevel 7
You could write an ACL (EXOS) or policy (EOS&EXOS) to drop any IPv6 packets at the edge ports by filtering on the IPv6 Ethertype of 0x86DD.

But this is a problem with the client devices (desktop systems in the recent case, printers in the older one) that should be addresses there.

BTW, since IPv6 uses multicast, non-IPv6 devices are usually less affected by misbehaving end systems than was (still is) the case with IPv4.
Userlevel 3
Hi, the fix for the root cause is to update the Intel nic drivers on the affected machines.

Reply