This website uses cookies. By clicking Accept, you consent to the use of cookies. Click Here to learn more about how we use cookies.
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- Extreme Networks
- Community List
- Switching & Routing
- ExtremeSwitching (EXOS/Switch Engine)
- RE: Link Problems, Flapping, Flood Rate Limit Acti...
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Link Problems, Flapping, Flood Rate Limit Activated, clear eee stats Feature unavailable?
Link Problems, Flapping, Flood Rate Limit Activated, clear eee stats Feature unavailable?
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-12-2015 09:53 PM
Have created an ACL that is meant to be blocking MDNS multicast addresses and an additional address used my Microsoft.
Have written the ACL to every port on Ingress so that I can see hits per port.
Problem is I'm not seeing the counters incrementing and aside from a packet trace I am confident there is this traffic on the network. I know this because we are trying to resolve an issue with a stack of X440's that keep rebooting because the CPU seems to be getting overwhelmed with packets from these address - as diagnosed by GTAC.
Policies at Policy Server:
Policy: Block_MDNS_Ingress
entry Block_1_MDNS_Ingress {
if match all {
source-address 224.0.0.251/32 ;
}
then {
deny ;
packet-count Block_251_MDNS_Ingress ;
}
}
entry Block_2_MDNS_Ingress {
if match all {
source-address 224.0.0.252/32 ;
}
then {
deny ;
packet-count Block_252_MDNS_Ingress ;
}
}
entry Block_3_MDNS_Ingress {
if match all {
source-address 239.255.255.250/32 ;
}
then {
deny ;
packet-count Block_250_MDNS_Ingress ;
}
}
Number of clients bound to policy: 1
Client: acl bound once
System Type: X440-48p (Stack)
SysHealth check: Enabled (Normal)
Recovery Mode: All
System Watchdog: Enabled
Current Time: Sat Sep 12 16:28:48 2015
Timezone: [Auto DST Disabled] GMT Offset: 0 minutes, name is UTC.
Boot Time: Fri Aug 28 00:37:38 2015
Boot Count: 135
Next Reboot: None scheduled
System UpTime: 15 days 15 hours 51 minutes 9 seconds
Slot: Slot-1 * Slot-2
------------------------ ------------------------
Current State: MASTER BACKUP (In Sync)
Image Selected: secondary secondary
Image Booted: secondary secondary
Primary ver: 15.3.1.4 15.3.1.4
Secondary ver: 15.5.4.2 15.5.4.2
patch1-5 patch1-5
Config Selected: primary.cfg
Config Booted: Factory Default
primary.cfg Created by ExtremeXOS version 15.5.4.2
2246563 bytes saved on Fri Sep 11 07:54:18 2015
Many thanks in advance.
Have written the ACL to every port on Ingress so that I can see hits per port.
Problem is I'm not seeing the counters incrementing and aside from a packet trace I am confident there is this traffic on the network. I know this because we are trying to resolve an issue with a stack of X440's that keep rebooting because the CPU seems to be getting overwhelmed with packets from these address - as diagnosed by GTAC.
Policies at Policy Server:
Policy: Block_MDNS_Ingress
entry Block_1_MDNS_Ingress {
if match all {
source-address 224.0.0.251/32 ;
}
then {
deny ;
packet-count Block_251_MDNS_Ingress ;
}
}
entry Block_2_MDNS_Ingress {
if match all {
source-address 224.0.0.252/32 ;
}
then {
deny ;
packet-count Block_252_MDNS_Ingress ;
}
}
entry Block_3_MDNS_Ingress {
if match all {
source-address 239.255.255.250/32 ;
}
then {
deny ;
packet-count Block_250_MDNS_Ingress ;
}
}
Number of clients bound to policy: 1
Client: acl bound once
System Type: X440-48p (Stack)
SysHealth check: Enabled (Normal)
Recovery Mode: All
System Watchdog: Enabled
Current Time: Sat Sep 12 16:28:48 2015
Timezone: [Auto DST Disabled] GMT Offset: 0 minutes, name is UTC.
Boot Time: Fri Aug 28 00:37:38 2015
Boot Count: 135
Next Reboot: None scheduled
System UpTime: 15 days 15 hours 51 minutes 9 seconds
Slot: Slot-1 * Slot-2
------------------------ ------------------------
Current State: MASTER BACKUP (In Sync)
Image Selected: secondary secondary
Image Booted: secondary secondary
Primary ver: 15.3.1.4 15.3.1.4
Secondary ver: 15.5.4.2 15.5.4.2
patch1-5 patch1-5
Config Selected: primary.cfg
Config Booted: Factory Default
primary.cfg Created by ExtremeXOS version 15.5.4.2
2246563 bytes saved on Fri Sep 11 07:54:18 2015
Many thanks in advance.
14 REPLIES 14
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-15-2015 07:11 AM
Thanks for the info and will certainly look into that and great advise.
My thoughts though are that since I turned off flow-control the amount of ports showing packet drops has significantly grown, so perhaps it was these ports that where sending the pauses?
With that it mind is there anyway (aside from attaching a sniffer) that I could see what packets are being dropped on the ports, some examples could be:
At the moment both ends are set to auto-negotiate and they are both negotiating 100mb full. I want to try fixing both ends but currently looking at the issue remotely.
There seems no logical reason packets should be being dropped, there are no other errors (like CRC) and there is plenty of bandwidth, and there is no other device attached to the phone. I could increase the buffer size for QP6 as its currently set to default but I really shouldn't be getting any contention as the traffic is so low?
If I could see what's being dropped, then that might give me a clue as to what's happening.
Also appreciate if there is any equivalent to configuring Link Flap in EXOS?
Many thanks.
My thoughts though are that since I turned off flow-control the amount of ports showing packet drops has significantly grown, so perhaps it was these ports that where sending the pauses?
With that it mind is there anyway (aside from attaching a sniffer) that I could see what packets are being dropped on the ports, some examples could be:
- debug command that outputs drops
- turn on a filter that outputs drops
- tcpdump on a single port
- write an ACL for all traffic on port and log
At the moment both ends are set to auto-negotiate and they are both negotiating 100mb full. I want to try fixing both ends but currently looking at the issue remotely.
There seems no logical reason packets should be being dropped, there are no other errors (like CRC) and there is plenty of bandwidth, and there is no other device attached to the phone. I could increase the buffer size for QP6 as its currently set to default but I really shouldn't be getting any contention as the traffic is so low?
If I could see what's being dropped, then that might give me a clue as to what's happening.
Also appreciate if there is any equivalent to configuring Link Flap in EXOS?
Many thanks.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-14-2015 06:54 PM
I know sleep has been touched on earlier in the thread - but end stations in power down mode often drop to 10Mb hdx and are infamous for negative network impact.
1) continuous pause frames (impact can go far beyond the local link)
2) high bandwidth ipv6 ND
These often hit a stack in distributed fashion since
* enterprises often use the same hardware and OS/drivers cookie cutter style
* a new version of a popular OS gets released and finds its way to many users PCs.
* at 5PM or whenever work or school gets out multiple end stations sleep at the same time
No docs on the pause behavior but see multicast ipv6 packet blurb here:
https://communities.intel.com/thread/48051.
http://community.spiceworks.com/topic/422869-dell-optiplex-9020-blasting-icmpv6-multicast-listener-d...
If broadcast or flood related, add wireshark to your growing list of analysis tools.
plenty of sound direction here already - but when there's free time try connecting to a troubled port with a wireshark laptop or other capture tool - wide open. Buffer will probably fill fast so a short snapshot is best - couple seconds. Capture may provide some insight.
Regards,
Mike
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-14-2015 10:44 AM
Enabled LLDP on 100mb port so that I could get details of Phone:
LLDP Port 1:5 detected 1 neighbor
Neighbor: (5.1)172.17.255.35/70:38:EE:D2:36:C3, age 11 seconds
- Chassis ID type: Network address (5); Address type: IPv4 (1)
Chassis ID : 172.17.255.35
- Port ID type: MAC address (3)
Port ID : 70:38:EE:D2:36:C3
- Time To Live: 120 seconds
- System Name: "AVXD236C3"
- System Capabilities : "Telephone"
Enabled Capabilities: "Telephone"
- Management Address Subtype: IPv4 (1)
Management Address : 172.17.255.35
Interface Number Subtype : System Port Number (3)
Interface Number : 1
Object ID String : "1.3.6.1.4.1.6889.1.69.3.1"
- IEEE802.3 MAC/PHY Configuration/Status
Auto-negotiation : Supported, Enabled (0x03)
Operational MAU Type : 100BaseTXHD (15)
- MED Capabilities: "MED Capabilities, Network Policy, Extended Power via MDI - PD, Inventory"
MED Device Type : Endpoint Class III (3)
- MED Extended Power-via-MDI
Power Type : PD Device (1)
Power Source : Unknown (0)
Power Priority: High (2)
Power Value : 5.1 Watts
- MED Network Policy
Application Type : Voice (1)
Policy Flags : Known Policy, Untagged (0x0)
VLAN ID : 200
L2 Priority : 6
DSCP Value : 46
- MED Network Policy
Application Type : Voice Signaling (2)
Policy Flags : Known Policy, Untagged (0x0)
VLAN ID : 200
L2 Priority : 6
DSCP Value : 46
- MED Hardware Revision: "1603D02A"
- MED Firmware Revision: "hb1603ua1_350B.bin"
- MED Software Revision: "ha1603ua1_350B.bin"
- MED Serial Number: "12WZ274604JK"
- MED Manufacturer Name: "Avaya"
- MED Model Name: "1603"
- Avaya/Extreme Conservation Level Support
Current Conservation Level: 0
Typical Power Value : 4.4 Watts
Maximum Power Value : 5.1 Watts
Conservation Power Level : 1=3.8W
- Avaya/Extreme Call Server(s): 172.17.255.240
- Avaya/Extreme IP Phone Address: 172.17.255.35 255.255.254.0
Default Gateway Address : 172.17.254.1
- Avaya/Extreme File Server(s): 0.0.0.0
- Avaya/Extreme IEEE 802.1q Framing: Tagged
Might see if I get the port set to auto-negotiate disable and see what happens?
What's also interesting is that when doing a 'show port congestion' all the ports increment drops packets in the same time frame (i.e wont increment for a few seconds, then all at the same time) and increment at the same rate?
LLDP Port 1:5 detected 1 neighbor
Neighbor: (5.1)172.17.255.35/70:38:EE:D2:36:C3, age 11 seconds
- Chassis ID type: Network address (5); Address type: IPv4 (1)
Chassis ID : 172.17.255.35
- Port ID type: MAC address (3)
Port ID : 70:38:EE:D2:36:C3
- Time To Live: 120 seconds
- System Name: "AVXD236C3"
- System Capabilities : "Telephone"
Enabled Capabilities: "Telephone"
- Management Address Subtype: IPv4 (1)
Management Address : 172.17.255.35
Interface Number Subtype : System Port Number (3)
Interface Number : 1
Object ID String : "1.3.6.1.4.1.6889.1.69.3.1"
- IEEE802.3 MAC/PHY Configuration/Status
Auto-negotiation : Supported, Enabled (0x03)
Operational MAU Type : 100BaseTXHD (15)
- MED Capabilities: "MED Capabilities, Network Policy, Extended Power via MDI - PD, Inventory"
MED Device Type : Endpoint Class III (3)
- MED Extended Power-via-MDI
Power Type : PD Device (1)
Power Source : Unknown (0)
Power Priority: High (2)
Power Value : 5.1 Watts
- MED Network Policy
Application Type : Voice (1)
Policy Flags : Known Policy, Untagged (0x0)
VLAN ID : 200
L2 Priority : 6
DSCP Value : 46
- MED Network Policy
Application Type : Voice Signaling (2)
Policy Flags : Known Policy, Untagged (0x0)
VLAN ID : 200
L2 Priority : 6
DSCP Value : 46
- MED Hardware Revision: "1603D02A"
- MED Firmware Revision: "hb1603ua1_350B.bin"
- MED Software Revision: "ha1603ua1_350B.bin"
- MED Serial Number: "12WZ274604JK"
- MED Manufacturer Name: "Avaya"
- MED Model Name: "1603"
- Avaya/Extreme Conservation Level Support
Current Conservation Level: 0
Typical Power Value : 4.4 Watts
Maximum Power Value : 5.1 Watts
Conservation Power Level : 1=3.8W
- Avaya/Extreme Call Server(s): 172.17.255.240
- Avaya/Extreme IP Phone Address: 172.17.255.35 255.255.254.0
Default Gateway Address : 172.17.254.1
- Avaya/Extreme File Server(s): 0.0.0.0
- Avaya/Extreme IEEE 802.1q Framing: Tagged
Might see if I get the port set to auto-negotiate disable and see what happens?
What's also interesting is that when doing a 'show port congestion' all the ports increment drops packets in the same time frame (i.e wont increment for a few seconds, then all at the same time) and increment at the same rate?
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-14-2015 09:01 AM
Hi,
Info below:
Stack 1.1 # show config | include eee
Stack 1.2 # show slot
Slots Type Configured State Ports
--------------------------------------------------------------------
Slot-1 X440-48p X440-48p Operational 48
Slot-2 X440-48p X440-48p Operational 48
Slot-3 X440-48p X440-48p Operational 48
Slot-4 X440-48p X440-48p Operational 48
Slot-5 Empty 0
Slot-6 Empty 0
Slot-7 Empty 0
Slot-8 Empty 0
Turned off flow control on all ports rx and tx and it made port congestion (packet drops) worse.
Stack 1.2 # show port congestion no-refresh
Port Congestion Monitor
Port Link Packet
State Drop
================================================================================
1:1 A 46853
1:5 A 46836
1:7 A 46836
1:13 A 46851
1:21 A 46850
1:26 A 34991
1:27 A 46838
1:32 A 46835
1:36 A 46836
1:38 A 46836
1:39 A 46836
1:41 A 46836
2:1 A 46846
2:3 A 46840
2:5 A 46840
2:14 A 46836
2:15 A 46836
2:17 A 46844
2:23 A 46842
2:26 A 55801
2:31 A 46836
2:32 A 46829
2:35 A 46845
2:38 A 46856
2:41 A 46839
2:43 A 46833
2:45 A 46825
3:1 A 46861
ACU A 46859
3:5 A 46864
3:9 A 46865
3:21 A 46867
3:27 A 46853
3:32 A 46868
3:34 A 46868
3:35 A 46868
3:43 A 46862
3:45 A 46868
3:46 A 46866
3:47 A 46868
4:1 A 46851
4:2 A 46842
4:5 A 46858
4:8 A 46855
4:11 A 46858
4:18 A 46865
4:19 A 46847
4:23 A 46861
4:39 A 46844
4:44 A 46869
================================================================================
Interesting is that all these ports are 100mb ports. So turned off negotiation and fixed speed and duplex but still getting packet drops! The other interesting thing is looking at the MAC addresses on these ports they all connected to Avaya IP phones - need to check the physical port settings.....
Here is the result of the debug hal show congestion when it entered it multiple times:
Congestion information for slot 1 type X440-48p since last query
Switch fabric congestion present: 7174025
Congestion information for slot 2 type X440-48p since last query
Switch fabric congestion present: 34594
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
Switch fabric congestion present: 34004
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
Info below:
Stack 1.1 # show config | include eee
Stack 1.2 # show slot
Slots Type Configured State Ports
--------------------------------------------------------------------
Slot-1 X440-48p X440-48p Operational 48
Slot-2 X440-48p X440-48p Operational 48
Slot-3 X440-48p X440-48p Operational 48
Slot-4 X440-48p X440-48p Operational 48
Slot-5 Empty 0
Slot-6 Empty 0
Slot-7 Empty 0
Slot-8 Empty 0
Turned off flow control on all ports rx and tx and it made port congestion (packet drops) worse.
Stack 1.2 # show port congestion no-refresh
Port Congestion Monitor
Port Link Packet
State Drop
================================================================================
1:1 A 46853
1:5 A 46836
1:7 A 46836
1:13 A 46851
1:21 A 46850
1:26 A 34991
1:27 A 46838
1:32 A 46835
1:36 A 46836
1:38 A 46836
1:39 A 46836
1:41 A 46836
2:1 A 46846
2:3 A 46840
2:5 A 46840
2:14 A 46836
2:15 A 46836
2:17 A 46844
2:23 A 46842
2:26 A 55801
2:31 A 46836
2:32 A 46829
2:35 A 46845
2:38 A 46856
2:41 A 46839
2:43 A 46833
2:45 A 46825
3:1 A 46861
ACU A 46859
3:5 A 46864
3:9 A 46865
3:21 A 46867
3:27 A 46853
3:32 A 46868
3:34 A 46868
3:35 A 46868
3:43 A 46862
3:45 A 46868
3:46 A 46866
3:47 A 46868
4:1 A 46851
4:2 A 46842
4:5 A 46858
4:8 A 46855
4:11 A 46858
4:18 A 46865
4:19 A 46847
4:23 A 46861
4:39 A 46844
4:44 A 46869
================================================================================
Interesting is that all these ports are 100mb ports. So turned off negotiation and fixed speed and duplex but still getting packet drops! The other interesting thing is looking at the MAC addresses on these ports they all connected to Avaya IP phones - need to check the physical port settings.....
Here is the result of the debug hal show congestion when it entered it multiple times:
Congestion information for slot 1 type X440-48p since last query
Switch fabric congestion present: 7174025
Congestion information for slot 2 type X440-48p since last query
Switch fabric congestion present: 34594
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
Switch fabric congestion present: 34004
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
----------------------------------------------------------------
Stack 1.2 # debug hal show congestion
Congestion information for slot 1 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 2 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 3 type X440-48p since last query
No switch fabric and CPU congestion present
Congestion information for slot 4 type X440-48p since last query
No switch fabric and CPU congestion present
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Report Inappropriate Content
09-14-2015 08:11 AM
Regarding the fabric congestion, please execute the debug hal show congestion output multiple times and check if that counter is incrementing in real time.
