purview receives netflow and mirrorN, but apps are not identified


Userlevel 3
  • New Member
  • 65 replies
Hi,
I installed a virtual vmware purview/analytics AAE in deployment mode 2 (Interface mirrored), with eth0 for netflow and management and eth1 for mirror packet reception:
0. Accept settings and continue
1. Hostname: interpur
2. Deployment Mode: Dual Interface Mirrored
3. Management Interface Configuration (eth0):
Address: 192.168.64.220
Netmask: 255.255.254.0
Gateway: 192.168.64.1
Nameserver: [our dns]
Domain name: [our domain]
4. NIS Server/Domain: Not Configured
5. Monitor Interface Configuration :
Tap Mode Interfaces eth1
[/code]An SSA switch sends its netflow packets towards 192.168.64.220 and has a mirror-n towards one of its switch ports which is directly connected to the AAE's host. (AAE's eth1 is mapped to a dedicated vswitch with promiscuous mode and All VLAN (4095) enabled. This vswitch uses the esxi-host's physical adpater which is directly connected to the SSA mirror port ge.1.17.)

SSA mirror and netflowconfig:
# mirror
set mirror create 2
set mirror 2 mirrorN 15
set mirror ports ge.1.17 2
# policy
set policy profile 1 name Purviewmirror pvid-status enable pvid 4095 mirror-destination 2
set policy rule admin-profile port lag.0.4 mask 16 port-string lag.0.4 admin-pid 1
set policy rule admin-profile port lag.0.5 mask 16 port-string lag.0.5 admin-pid 1
set policy rule admin-profile port ge.1.45 mask 16 port-string ge.1.45 admin-pid 1
set policy rule admin-profile port tg.1.4 mask 16 port-string tg.1.4 admin-pid 1
# netflow
set netflow export-interval 1
set netflow export-destination 192.168.64.220 2055
set netflow export-version 9
set netflow port lag.0.4-5 enable rx
set netflow port ge.1.45 enable rx
set netflow port tg.1.4 enable rx
set netflow template refresh-rate 30 timeout 1
set netflow cache enable[/code]AAE receives netflow packets on eth0:
root@interpur:~$ tcpdump -i eth0 udp port 2055
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:28:35.047775 IP 192.168.64.9.2055 > interpur.man.htw-berlin.de.2055: UDP, length 1420
11:28:35.052489 IP 192.168.64.9.2055 > interpur.man.htw-berlin.de.2055: UDP, length 1444
11:28:35.058061 IP 192.168.64.9.2055 > interpur.man.htw-berlin.de.2055: UDP, length 1464
...[/code]AAE receives mirror packets on eth1:
[/code]root@interpur:~$ tcpdump -i eth1 -c 2
tcpdump: WARNING: eth1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
11:32:07.549780 20:b3:99🇦🇩72:3b (oui Unknown) > 20:b3:99:55:a0:1f (oui Unknown), ethertype Unknown (0x7034), length 70:
0x0000: a001 0800 4500 0034 01b2 4000 3406 395e ....E..4..@.4.9^
0x0010: 40ca 7004 8d2d cdb8 0050 c52b 245a cadb @.p..-...P.+$Z..
0x0020: 12e4 469b 8010 003e e21e 0000 0101 080a ..F....>........
0x0030: 6b12 fa03 36f0 de74 k...6..t
11:32:07.549789 20:b3:99🇦🇩72:3b (oui Unknown) > 20:b3:99:55:a0:1f (oui Unknown), ethertype Unknown (0x7034), length 64:
0x0000: a001 0800 4500 002c 104b 0000 3006 d058 ....E..,.K..0..X
0x0010: bad4 81d1 8d2d e055 0aac 0913 8d2d e055 .....-.U.....-.U
0x0020: 0000 0000 6002 11b4 5b0f 0000 0204 05ac ....'...[.......
0x0030: 0000 ..[/code]
Management Center also shows netflow and mirror packets:



But the Identification Rate stays at 0% and Application Infos are not populated. What could be the reason? AAE was enforced and rebooted.

20 replies

Userlevel 3
Hi, do you have enabled jumbo frames on all the path between the sensor and the engine?
Userlevel 3
Hi,
on SSA-Switch "show port jumbo ge.1.17" shows jumbo is adminstratively enabled and operational with MTU 10239.
Userlevel 3
Ok, ensure that jumbo is enabled along all the path...then you need to wait for a while before the engine collects the flows and display it on the ExtremeControl...
Userlevel 3
Jumbo was enabled from the beginning on this switch port, so there is nothing to wait for. The path is quite short becauase the VMs tap-port vswitch is directly connected to the switch port.

edit:
Netsight Console Alarm says: "No bidirectional traffic seen on interface eth1". It seems as if the appliance gets mirror packets, but does not recognize them.
Userlevel 2
Hi.

configure ge.1.17 for 1000 Full Duplex.

set port duplex ge.1.17 full

set port speed ge.1.17 1000

regards

Alex
Userlevel 5
Make sure you set the "location" setting to include the subnet that you are monitoring. Without that you will not get ANY identification.

Userlevel 5
Under Analytics, go to configuration, then to location.Make sure the "location" is set with the IP range that you are monitoring.

Userlevel 3
Hi,
1000 full duplex was already configured.

edit: Locations exist. (I already used a GRE-tunnel purview successfully with identified apps.)
Userlevel 6
1. In a 'show port status ge.1.17' the mirrorN port must be operationally and administratively up.
2. The packet capture should be run longer, and make sure that the traffic seen in both directions.

The policy and netflow portions look correct. But lets make sure that that eth1 is receiving bilateral communications from those ports.
Userlevel 3
Hi,
>show port status ge.1.17
Port Alias Oper Admin Speed Duplex Type
(truncated) Status Status (bps)
------------ ---------------- -------- ------- ------ ------- ------------------
ge.1.17 up up 1.0G full 1000-t rj45

[/code]The mirror setting definitively mirrors bilateral communications. tg.1.4 and ge.1.45 are ports towards external network and lag.0.4-5 are ports to internal networks. These ports do netflow and mirror ingress packets. A GRE-tunnel purview was able to decode bilateral traffic from that n-mirror. I cant paste much longer captures here as there are about 10,000 packets per second.
tcpdump -i eth1 -q
14:45:03.936153 20:b3:99🇦🇩72:3b (oui Unknown) > 00:16:9c:6e:2f:80 (oui Unknown), Unknown Ethertype (0x7034), length 74:
14:45:03.936299 20:b3:99🇦🇩72:3b (oui Unknown) > 20:b3:99:55:a0:1f (oui Unknown), Unknown Ethertype (0x7034), length 70:
14:45:03.936420 20:b3:99🇦🇩72:3b (oui Unknown) > 20:b3:99:55:a0:1f (oui Unknown), Unknown Ethertype (0x7034), length 64:
14:45:03.936425 20:b3:99🇦🇩72:3b (oui Unknown) > 00:16:9c:6e:2f:80 (oui Unknown), Unknown Ethertype (0x7034), length 77:
14:45:03.936552 20:b3:99🇦🇩72:3b (oui Unknown) > 20:b3:99:55:a0:1f (oui Unknown), Unknown Ethertype (0x7034), length 99:
14:45:03.936602 20:b3:99🇦🇩72:3b (oui Unknown) > 00:16:9c:6e:2f:80 (oui Unknown), Unknown Ethertype (0x7034), length 307:
14:45:03.936728 20:b3:99🇦🇩72:3b (oui Unknown) > 00:16:9c:6e:2f:80 (oui Unknown), Unknown Ethertype (0x7034), length 965:
14:45:03.936850 20:b3:99🇦🇩72:3b (oui Unknown) > 00:16:9c:6e:2f:80 (oui Unknown), Unknown Ethertype (0x7034), length 171:

[/code]I am wondering why the dump from eth1 says "oui unknown" and "ethertype 0x7034" in EVERY packet. This doesnt look normal. Maybe something offsets the packets so AEE cant decode them?
Userlevel 6
This appears to be one way traffic - 0x7034 appears to be CDP/EDP Discovery protocol packets. So I think your just seeing passive data from attached S-Series port. I dont think data is being mirrored properly from the S-Series.
I would stick a network analyzer on the mirror port directly (ge.1.17) to see if it is forwarding two way traffic, or if the problem is localized possibly to V-Switch.
Userlevel 3
Hi,
This is a wireshark trace directly from the n-mirror port:


F.e. we see a paket here which has something to to with SIP. We also see the 0800 ethertype at byte 16 and 17 but it is at the wrong postion. Shouldnt it be at byte 12-13? I dont know where 70 34 and a0 01 do come from. They are in every packet.

Dropboxlink to that pcap-file (about 10 MB): https://www.dropbox.com/s/2tmwfgbyui1tz1k/purviewmirror.pc.png?dl=0

Ethertype 0800 or 8100 (for tagged) is always at byte 16 and 17 in the captures.

SSA-G8018-0652 FW:08.62.01.0034
Userlevel 6
Where was the trace taken?
Userlevel 3
A computer with windows wireshark was directly connected to SSA ge.1.17.
Userlevel 6
I would open a GTAC ticket, it does not appear to be an obvious problem.

https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-contact-Extreme-Networks-Global-Tec...
I notice I have these two lines in my config:

set netflow export-data enable mac
set netflow export-data enable vlan
Userlevel 3
Since it is unlikely that policy mirroring is totally broken I sent the mirror towards another switch port.
set mirror ports ge.1.19 2[/code]Now it worked, packets were clearly decodable. 🙂

When I configure the mirror back to ge.1.17 the problem occurs again. So the problem sticks with the switch port. Maybe it is because before trying the policy mirror config, ge.1.17 was used as target port for a GRE-Tunnel before ("tunnel mode gre l2 ge.1.17" in tun-interface definition). The GRE-tunnel mirror worked perfectly. When I started to change to policy mirror configuration I deleted the loopback and tunnel-interfaces. But maybe switch port ge.1.17 still sticks somehow in that mode. I can reboot the router at night to see if ge.1.17 gets "normal" after a reboot.
Userlevel 7
htw wrote:

Since it is unlikely that policy mirroring is totally broken I sent the mirror towards another switch port.
set mirror ports ge.1.19 2[/code]Now it worked, packets were clearly decodable. 🙂

When I configure the mirror back to ge.1.17 the problem occurs again. So the problem sticks with the switch port. Maybe it is because before trying the policy mirror config, ge.1.17 was used as target port for a GRE-Tunnel before ("tunnel mode gre l2 ge.1.17" in tun-interface definition). The GRE-tunnel mirror worked perfectly. When I started to change to policy mirror configuration I deleted the loopback and tunnel-interfaces. But maybe switch port ge.1.17 still sticks somehow in that mode. I can reboot the router at night to see if ge.1.17 gets "normal" after a reboot.

Since the packets seemed to be encapsulated according to your packet traces, this seems plausible.
Userlevel 3
Hi,
after a reboot the affected port returned to normal policy mirror behaviour too.
Userlevel 4
Worth Noting, on the netflow. The reason it's reporting no bi-rdirectional because the netflow config is only sending RX.|

Should look something like "set netflow port ge.1.45 enable both"
Userlevel 3
Hi Thomas,
in this case "netflow enable rx" is not a problem. Seeing this SSA as a border router between two networks, ge.1.45 and tg.1.4 are "outside" and the two LAGs are "inside". All of them are set to rx, so traffic getting into the net domain and flowing out of the domain gets detected. When the appliance startet to decodes the traffic properly, the "no bidirectional traffic"-message disappeared.

Purview Deployment guide suggest setting netflow to rx on inside and outside interface f.e. to prevent double netflow detection and since policy mirroring works only ingress too. I wonder if one can combine the ingress mirrors on insinde and outside with a "netflow both" on the inside for Purview detection. hmmm...

Reply