X590 unable to ping certain host

[msuggs]

Hello, 

i really need some help here. i am replacing a x460 stack with a x590 stack, i cut in new 590 stack but was unable to ping some gear on a subnet. very odd issue here. i have 66 vlans, loaded with traffic, i am using OSPF. PIM  and L2 routing where and how we want network to run. so after cutting to the 590’s i notice my Adtran equipment a DSLAM could not respond to a Adtran Mgmt server in another exchange.

VR-Default    10.100.144.10    20:47:47:7a:82:54    3      NO  Adtran-Mdvl-Mgmt  144   4:6
VR-Default    10.100.144.50    00:a0:c8:7a:05:3e   13      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.51    00:a0:c8:78:47:09    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.60    00:a0:c8:81:56:46    3      NO  Adtran-Mdvl-Mgmt  144   2:20
VR-Default    10.100.144.70    00:a0:c8:7b:ad:f5    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.80    00:a0:c8:7a:02:5c    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.90    00:a0:c8:be:36:7d    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.100   00:a0:c8:d2:c8:40    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.110   00:19:92:e2:1a:d7    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.120   00:19:92:d4:d5:33    3      NO  Adtran-Mdvl-Mgmt  144   2:20
VR-Default    10.100.144.125   00:19:92:f2:6d:08    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.130   00:19:92:eb:f6:c8    3      NO  Adtran-Mdvl-Mgmt  144   2:24
VR-Default    10.100.144.135   00:19:92:eb:ed:6e    3      NO  Adtran-Mdvl-Mgmt  144   2:24
after modifying iproutes etc and trouble shooting, the end results is the cabinet with address 10.100.145.10 can’t ping 10.100.144.50.. very odd here that i can ping other host on 10.100.144.xx it is like the 10.100.144.50 was blacklisted. we dont have any ACL’s or any black list. when i cut back to the 460’s all traffic was fine. Ipforwarding is enabled globally. i worked at this pretty hard before cutting back to the 460’s. it makes no sense to me that the 590’s would not route the specific host address.  i just so happen to cut in another set of 590’s. there  i had to change my iproute from a mgmt subnet to a first hop subnet.. very odd. i also tried changing the route on this config and did not help any. i even just removed iprouting and went OSPF.

Adtran-NWAG-MGMT 145  10.100.145.1   /24  -f-------o-----------------

Adtran-Mdvl-Mgmt 144  10.100.144.1   /24  -f-------------

Meadville-8810.3 # sh iproute 10.100.145.0/24
Ori  Destination        Gateway         Mtr  Flags         VLAN       Duration
#oa  10.100.145.0/24    192.168.252.2   4    UG-D---um--f- Video-10Gig 14d:5h:23m:40s

in my network i have a 8806, 8810, 4 -460stacks.,, and a 590 stack all in a EAPS 10G ring.

590 sh switch

Primary ver:      31.1.1.3                     31.1.1.3
                  patch1-1                     patch1-1

8810 sh switch 

Primary ver:      16.2.5.4                     16.2.5.4
                  patch1-29                    patch1-29

460 sh switch

Primary ver:      22.7.3.5                     22.7.3.5
 

i have setup a LAB environment to duplicate the issue, from the ADTRAN to a PC sitting on a similar subnet i can ping the GW but not a PC.

MDVLHST-F01#ping 10.100.123.1
Pinging 10.100.123.1 (10.100.123.1):
Press ^C to Cancel
Reply from 10.100.123.1 (10.100.123.1): time=7ms
Reply from 10.100.123.1 (10.100.123.1): time=7ms
Reply from 10.100.123.1 (10.100.123.1): time=8ms
Reply from 10.100.123.1 (10.100.123.1): time=6ms

Ping statistics for 10.100.123.1 (10.100.123.1):
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 6ms, Maximum = 8ms, Average = 7ms
MDVLHST-F01#ping 10.100.123.2
Pinging 10.100.123.2 (10.100.123.2):
Press ^C to Cancel
Request timed out.
Request timed out.
Request timed out.
 

[Stefan_K_]

The screenshot is from the client with IP 10.100.123.3? Then it could be the following:

• windows sends the icmp reply via the wrong interface, which would explain why it isn’t in the wireshark log. Were you connected via cable and via wifi?
• windows firewall (or similar software) is blocking the icmp requests. This explains why it is seen on the interface, but since it doesn’t get processed any further, there is no reply

Best regards
Stefan

[msuggs]

Meadville-8810.10 # ping 10.100.123.3
Ping(ICMP) 10.100.123.3: 4 packets, 8 data bytes, interval 1 second(s).

--- 10.100.123.3 ping statistics ---
4 packets transmitted, 0 packets received, 100% loss
round-trip min/avg/max = 0/0/0 ms

wireshark shows a ping request 8810 does not see response.
 

[msuggs]

Thanks, Stefan for your reply, i have a open ticket with Adtran vendor asking that very question.

however i have a PC windows 10 attached to x590 10.100.123.3 i can ping PC from x590 but not from 8810.. very odd..

Meadville-8810.5 # traceroute 10.100.123.3
traceroute to 10.100.123.3, 30 hops max
 1  192.168.239.101                              1 ms        0 ms        1 ms
 2                                               *           *           *
 3                                               *           *           *^C

TRACERT GETS ME TO THE X590 SWITCH MGMT CONNECTION 192.168.239.101 BUT NOT TO TEST VLAN.

Name            VID  Protocol Addr       Flags                         Proto  Ports  Virtual
                                                                              Active router
                                                                              /Total
-----------------------------------------------------------------------------------------------
Backchannel     539  192.168.239.101/24  -f------------------------    ANY    1 /1   VR-Default
Default         1    --------------------------------T-------------    ANY    1 /70  VR-Default
Mgmt            4095 ----------------------------------------------    ANY    0 /1   VR-Mgmt
TEST-Adtran     4093 10.100.123.1   /24  -f------------------------    ANY    2 /2   VR-Default
 

* Slot-1 X590.14 # ping 10.100.123.3
Ping(ICMP) 10.100.123.3: 4 packets, 8 data bytes, interval 1 second(s).
16 bytes from 10.100.123.3: icmp_seq=0 ttl=128 time=0.734 ms
16 bytes from 10.100.123.3: icmp_seq=1 ttl=128 time=1.283 ms
16 bytes from 10.100.123.3: icmp_seq=2 ttl=128 time=0.700 ms
16 bytes from 10.100.123.3: icmp_seq=3 ttl=128 time=0.674 ms

 
* Slot-1 X590.12 # traceroute 10.100.123.3
traceroute to 10.100.123.3, 30 hops max
 1                                               *           *           *
 2                                               *           *           *
 3                                               *           *       ^C
 

Meadville-8810.6 # ping 10.100.123.1
Ping(ICMP) 10.100.123.1: 4 packets, 8 data bytes, interval 1 second(s).
16 bytes from 10.100.123.1: icmp_seq=0 ttl=64 time=13 ms
16 bytes from 10.100.123.1: icmp_seq=1 ttl=64 time=4.356 ms
 
 Meadville-8810.7 # ping 10.100.123.3
Ping(ICMP) 10.100.123.3: 4 packets, 8 data bytes, interval 1 second(s).

--- 10.100.123.3 ping statistics ---
4 packets transmitted, 0 packets received, 100% loss
 

THANKS,

[Stefan_K_]

This doesn’t have to be an issue with the X590 but can also be an issue with the end-system (10.100.144.50). Don’t know what kind of devices these are but:

• is it possible that there are static arp entries on this end-system? (for the default gateway; which resolves to the MAC-address of the X460)
• Can you use wireshark on the end-system? If not, can you do a port-mirror and use wireshark?
• Can you disconnect 10.100.144.50 from the network, give your client this IP-Address and then check if everything works? If yes, then you can be sure that the X590 does not “blacklist” this IP-address. 🙂 

Best regards
Stefan