Not all packets are forwarded

  • 0
  • 1
  • Problem
  • Updated 2 years ago
  • Solved
When I enable port mirroring on our Extreme Summit x460-24t not all the packets are sent to the mirror destination port. In a wireshark capture from the mirror port I only see very few (less than 1% of the actual) audio packets (ieee1722.cdfield == 0). When doing a packet capture using another (non-extreme) switch I see all the packets.

To enable port mirroring I used these commands:

# enable mirror lavb to port 1
# config mirror lavb add port 9-15
# show mirror

DefaultMirror   (Disabled)
    Description:    Default Mirror Instance, created automatically
    Mirror to port: 1
    Source filters configured :
        Ports 9-24, all vlans, ingress and egress

lavb   (Enabled)
    Description:   
    Mirror to port: 1
    Source filter instances used :  7
    Port 9, all vlans, ingress and egress
    Port 10, all vlans, ingress and egress
    Port 11, all vlans, ingress and egress
    Port 12, all vlans, ingress and egress
    Port 13, all vlans, ingress and egress
    Port 14, all vlans, ingress and egress
    Port 15, all vlans, ingress and egress

Mirrors defined:          2
Mirrors enabled:          1 (Maximum 4)
HW filter instances used: 7 (Maximum 128)
HW mirror instances used: 1 ingress, 1 egress (Maximum 4 total, 2 egress)

Switch version:
* X460-24p.2 # show version
Switch      : 800323-00-05 1120G-81240 Rev 5.0 BootROM: 2.0.1.7    IMG: 16.1.1.4 
PSU-1       : PSSF751301A-11 800382-00-04 1120A-40142
PSU-2       :

Image   : ExtremeXOS version 16.1.1.4 16.1.1.4-patch1-1 by release-manager
          on Thu Aug 6 16:14:58 EDT 2015
BootROM : 2.0.1.7
Diagnostics : 6.3

Am I missing something in the configuration? Each of those ports has a maximum of 18Mbps traffic so the total traffic should be much less than the 1Gbps allowed by the port being mirrored to.
Photo of Leon Wessels

Leon Wessels

  • 210 Points 100 badge 2x thumb

Posted 2 years ago

  • 0
  • 1
Photo of Bill Stritzinger

Bill Stritzinger, Alum

  • 6,016 Points 5k badge 2x thumb
Leon, 

Your configuration looks OK.  Have you checked the drivers of the NIC on the laptop or machine running wireshark?  I have seen where issues on the drivers prevents sometimes packets with tags and other strange things and/or traffic not being seen.  Check that and let us know...

Bill
Photo of Leon Wessels

Leon Wessels

  • 210 Points 100 badge 2x thumb
Hi

I'm using Ubuntu 14.04 with kernel 3.13.0-85-generic.
I tried these three network cards (output from lspci)
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM57762 Gigabit Ethernet PCIe
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

I also tried the following on the pc:
# sudo vconfig add eth2 2
# sudo cat /proc/net/vlan/config
VLAN Dev name     | VLAN ID
Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
eth2.2         | 2  | eth2
# sudo cat /proc/net/vlan/eth2.2
eth2.2  VID: 2     REORDER_HDR: 1  dev->priv_flags: 1
         total frames received         1166
          total bytes received       261184
      Broadcast/Multicast Rcvd         1166

      total frames transmitted            0
       total bytes transmitted            0
Device: eth2
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings:

On the switch it seems that the avb packets have vlan tag 2.
# show vlan SYS_VLAN_0002
VLAN Interface with name SYS_VLAN_0002 created dynamically
    Admin State:     Enabled     Tagging:    802.1Q Tag 2
    Description:     None
    Virtual router:     VR-Default
    IPv4 Forwarding:     Disabled
    IPv4 MC Forwarding:  Disabled
    IPv6 Forwarding:     Disabled
    IPv6 MC Forwarding:  Disabled
    IPv6:                None
    STPD:               s0(Disabled)
    Protocol:            Match all unfiltered protocols
    Loopback:            Disabled
    NetLogin:            Disabled
    OpenFlow:            Disabled
    TRILL:               Disabled
    QosProfile:          None configured
    Flood Rate Limit QosProfile:       None configured
    Ports:   15.       (Number of active ports=15)
       Tag:       *5H,    *9H,   *11H,   *13H,   *14H,   *16H,   *17H,
                 *20H,   *21H,   *22H,   *23H,   *24H

None of this made any difference. Is there any way to forward the packets without the vlan tag?

This command:
# enable mirror lavb to port 1

Produces this output:
WARNING: This command will remove VLAN membership from the monitor port.
         Do you want to continue? (y/N) Yes

Should I be concerned about it at all?

Regards
Leon
Photo of Derek Bird

Derek Bird, Employee

  • 576 Points 500 badge 2x thumb
Leon:

It's important that promiscuous mode is enabled on the interface you're capturing with.  Here's what you're looking for in capture options in Wireshark:

Photo of Leon Wessels

Leon Wessels

  • 210 Points 100 badge 2x thumb
I checked and promiscuous mode was enabled.
Photo of Martin Flammia

Martin Flammia

  • 5,744 Points 5k badge 2x thumb
Perhaps do a 'show ports congestion', see if any packets are being dropped?
Photo of Leon Wessels

Leon Wessels

  • 210 Points 100 badge 2x thumb
Thanks for the tip. This seems to be the problem. About 7000 packets are dropped per second. I changed my mirror configuration so that I mirror a single port and the number of packets dropped per second seems to be about the same. I don't know how to fix this though. I don't see any reason why the packets should be dropped.

Output that might be useful to debug the dropped packets:
 # show port 1 buffer
Packet Buffer Allocation for ports in range 1-30
 Total Packet Buffer Size: 3145728 bytes, Not Overcommitted
 Total Shared Buffer Size: 2850304
  Port 1  Max Shared Buffer Usage: 569984 bytes (20%)
   QP1: Reserved Buffer: 1536 bytes
   QP6: Reserved Buffer: 1536 bytes
   QP7: Reserved Buffer: 1536 bytes
   QP8: Reserved Buffer: 1536 bytes

# show port 1 configuration
Port Configuration Monitor                              Tue Apr 19 18:00:04 2016
Port     Virtual    Port  Link  Auto   Speed      Duplex   Flow  Load   Media
         router     State State Neg  Cfg Actual Cfg Actual Cntrl Master Pri Red
================================================================================
1        VR-Default  E      A    ON  AUTO  1000 AUTO FULL SY/ASY        UTP    

# show port 1 information
Port      Flags               Link      ELSM Link Num  Num   Num Jumbo QOS     Load
                              State     /OAM  UPS STP VLAN Proto  Size profile Master
=====================================================================================
1         Em------e--fMB----- active    - / -   0   0    0     0  9216 none   
=====================================================================================
> indicates Port Display Name truncated past 8 characters
Flags : a - Load Sharing Algorithm address-based,
        b - Rx and Tx Flow Control Enabled, B - Broadcast Flooding Enabled,
        D - Port Disabled, e - Extreme Discovery Protocol Enabled,
        E - Port Enabled, f - Unicast Flooding Enabled,
        F - Priority Flow Control Enabled, G - MLAG Enabled, i - Isolation,
        j - Jumbo Frame Enabled, l - Load Sharing Enabled,
        L - Extreme Link Status Monitoring Enabled,
        m - MACLearning Enabled, M - Multicast Flooding Enabled,
        n - Ingress TOS Enabled, o - Dot1p Replacement Enabled,
        O - Ethernet OAM Enabled, p - Load Sharing Algorithm port-based,
        P - Software redundant port(Primary),
        R - Software redundant port(Redundant), s - diffserv Replacement Enabled,
        v - Vman Enabled, w - MACLearning Disabled with Forwarding,
        x - Rx Flow Control Enabled

//the counters of both ports were reset at the same time.
# show port 1,9 packet
Port Packet Statistics                                  Tue Apr 19 18:05:05 2016
Port      Link                         Packet Sizes
          State   0-64    65-127   128-255   256-511  512-1023 1024-1518   Jumbo
================================================================================
1         A         8       198        79         1         0         0        0
9         A        71       121     32346         1         0         0        0

For reference this is the new mirror setup:
#show mirror

DefaultMirror   (Disabled)
    Description:    Default Mirror Instance, created automatically
    Mirror to port: -

lavb   (Enabled)
    Description:   
    Mirror to port: 1
    Source filter instances used :  1
    Port 9, all vlans, ingress and egress

Mirrors defined:          2
Mirrors enabled:          1 (Maximum 4)
HW filter instances used: 1 (Maximum 128)
HW mirror instances used: 1 ingress, 1 egress (Maximum 4 total, 2 egress)
Photo of Leon Wessels

Leon Wessels

  • 210 Points 100 badge 2x thumb
Thanks to all for the help. The dropped packets seem to be caused when avb is enabled on the port being mirrored to. Disabling avb on that port before enabling the mirroring fixes this issue.

For future reference have a look at this article:
https://gtacknowledge.extremenetworks.com/articles/Solution/AVB-Mirroring-AVB-Control-Protocol-Traff...