5520 switch, 1gbps optics in VIM, connected but no IP connectivity

[Keith9]

I have a new 2 switch stack of 5520’s each with VIM-4X in the right side.  I have 2 10gbps fibers that work perfectly in a load sharing to connect to another floor, and 2 Active Twinax to a vmware server in a load sharing link working perfectly.

I have a wireless rooftop backup antenna, its only backup so not mission critical as I have a 1gbps WAN and a 100mbps WAN already in this building, but I noticed no IP connectivity to this over the 1gbps fiber.

Port 1:58 shows 1000 FULL connection.  The port link lights are up.
The port is in untagged vlan P2P-WIRELESS, and has an ip of 192.168.4.2/28.

The radio on the roof has an ip of 192.168.4.5/28 and is NOT pingable.

Neither is the other side of the link 192.168.4.1/28 at our HQ building about 500 feet away.  No OSPF adjacency has formed becuase even though we have Layer 1 link, it doesn’t seem like traffic likes to pass on 1gbps SFP’s in VIM4X’s on 5520’s running EXOS 31.2.1.1.

sh iparp port 1:58 shows no mac entries,  sh nodealias ports 1:58 shows nothing either.

I did disable port 1:58 then enable port 1:58, in the log it shows this…

03/05/2021 10:30:50.85 <Info:vlan.msgs.portLinkStateUp> Port 1:58 link UP at speed 1 Gbps and full-duplex

So link up but IP traffic won’t traverse.

sh port 1:58 vid
         Untagged
Port     /Tagged   VID(s)
-------- --------  ------------------------------------------------------------
1:58     Untagged  44
vlan info

P2P-WIRELESS    44   192.168.4.2    /28  -f-------o----------------    ANY    1 /1   VR-Default

VLAN Interface with name P2P-WIRELESS created by user
    Admin State:         Enabled     Tagging:   802.1Q Tag 44
    Description:         None
    Virtual router:      VR-Default
    IP Anycast:          Disabled
    IPv4 Forwarding:     Enabled
    IPv4 MC Forwarding:  Disabled
    Primary IP:          192.168.4.2/28
    IPv6 Forwarding:     Disabled
    IPv6 MC Forwarding:  Disabled
    IPv6:                None
    STPD:                None
    Protocol:            Match all unfiltered protocols
    Loopback:            Disabled
    NetLogin:            Disabled
    QosProfile:          None configured
    Egress Rate Limit Designated Port: None configured
    Flood Rate Limit QosProfile:       None configured
    Suppress ARP:        Disabled
    Suppress ND:         Disabled
    Proxy ARP:           Entry required
    Ports:   1.           (Number of active ports=1)
       Untag:   *1:58
        Flags:    (*) Active, (!) Disabled, (g) Load Sharing port
                  (b) Port blocked on the vlan, (m) Mac-Based port
                  (i) Port inactivated on the vlan due to VXLAN configuration
                  (a) Egress traffic allowed for NetLogin
                  (u) Egress traffic unallowed for NetLogin
                  (t) Translate VLAN tag for Private-VLAN
                  (s) Private-VLAN System Port, (L) Loopback port
                  (x) VMAN Tag Translated port
                  (A) Dynamically added by Auto-peering
                  (F) Dynamically added by Fabric Attach
                  (G) Multi-switch LAG Group port
                  (H) Dynamically added by MVRP
                  (I) Dynamically added by IDM
                  (N) Dynamically added by Netlogin
                  (U) Dynamically added uplink port
                  (V) Dynamically added by VM Tracking
pinging the device on the roof (which obvisouly has power since the laser is working in the sfp...) times out.  Here’s the fiber signal levels for this 1gbps sfp:

sh port 1:58 transceiver information

Port      Temp    TxPower  RxPower  TxBiasCurrent  Voltage-Aux1/ Voltage-Aux2
       (Celsius)   (dBm)    (dBm)     (mA)         Vcc (Volts)       (Volts)
================================================================================
1:58      27.83    -4.56    -14.24    2.63          3.37          N/A
================================================================================
      N/A  - parameter not applicable to transceiver connected to the port
      D    - Optic supports Digital Diagnostic Monitoring Interface (DDMI) which is off
      *    - value is out of normal range
      -inf - negative infinity, parameter not defined
 

So how can one have layer 1 link but no IP connectivity?

The Cisco 3750X switch stack that was here previously had no problem with a Cisco SFP in sled Gi1/1/1.

Here is the previous Cisco config

interface GigabitEthernet1/1/1
 description LightPointe FSO Link
 switchport access vlan 44
 ip arp inspection trust
 ip access-group acl1 in
 mls qos trust dscp
 spanning-tree portfast edge
 ip dhcp snooping trust

Here’s everything in EXOS having to do with port 1:58
sh configuration | i 1:58
configure vlan default delete ports 1:2,1:45,1:47,1:58,2:48,2:57,2:59
enable diffserv examination port 1:58
disable dot1p examination port 1:58
configure ports 1:58 description-string "P2P Wireless connection to HQ VL44"
configure ports 1:58 auto off speed 1000 duplex full
configure vlan P2P-WIRELESS add ports 1:58 untagged
configure qosprofile QP3 minbw 0 maxbw 100 ports 1:58
configure qosprofile QP5 minbw 0 maxbw 100 ports 1:58
disable edp ports 1:58
enable cdp ports 1:58
enable nodealias ports 1:58

And for the vlan

sh configuration | i P2P-WIRELESS
create vlan "P2P-WIRELESS"
configure vlan P2P-WIRELESS tag 44
configure vlan P2P-WIRELESS add ports 1:58 untagged
configure vlan P2P-WIRELESS ipaddress 192.168.4.2 255.255.255.240
enable ipforwarding vlan P2P-WIRELESS
enable bootprelay ipv4 vlan P2P-WIRELESS
configure ospf add vlan P2P-WIRELESS area 0.0.0.0
configure ospf vlan P2P-WIRELESS cost 15
configure ospf vlan P2P-WIRELESS priority 0
configure ospf vlan P2P-WIRELESS authentication encrypted md5 1 "encrypted string"


This should be pretty cut and dry, shouldnt it?

[Keith9]

Yes,  5520 are only supported by 31.x. That’s why I mentioned that I hope that I won’t get in contact with 5520 in the near future. 😄 

According to https://optics.extremenetworks.com/ the I-MGBIC-GSX is officially compatible only with X440G2 and X435. (even though it works in other switches...)

What I didn’t understand: Have you tested an official supported SFP like 10051H in an 5520? If not, could you maybe test this (even if this doesn’t help in your case). I’m curious… 

I could get the FS.com Extreme coded version of the 10051H.  Otherwise overpriced SFP modules are not in the budget.

We can have a shoebox full of spares for the price of one “official” branded module.  Not worth it IMO.

I actually have an order to place today for some fiber optic cabling for another project.  I’ll throw one on, its only $7.

[Stefan_K_]

Yes,  5520 are only supported by 31.x. That’s why I mentioned that I hope that I won’t get in contact with 5520 in the near future. 😄 

According to https://optics.extremenetworks.com/ the I-MGBIC-GSX is officially compatible only with X440G2 and X435. (even though it works in other switches...)

What I didn’t understand: Have you tested an official supported SFP like 10051H in an 5520? If not, could you maybe test this (even if this doesn’t help in your case). I’m curious… 

[Keith9]

Extreme Networks I-MGBIC-GSX Compatible 1000BASE-SX SFP 850nm 550m DOM LC MMF Transceiver Module

https://www.fs.com/products/32864.html

These work perfectly in X690’s and X450G2’s, as well as other fiber optic class modules of various 1gbps wavelengths and part numbers like 10065, 10051H, etc.  Also 1000BASE-T such as https://www.fs.com/products/32882.html work perfectly in every Extreme switch model I’ve tried except the 5520’s VIM-4X slots.

The 5520 series switches don’t seem to pass ANY kind of IP traffic for regular SFP’s in the VIM-4X slots.  SFP+ like 10gig fiber at various wavelengths or TWINAX cables to servers running at 10gig all work fine.

Its not a port speed issue either.  I tried the whole auto off speed 1000 thing too.

They are running 31.2.1.1 which fixed the autonegotiation issue for 100mbps and slower devices on the front facing 10/100/1000 baseT RJ45 ports.

They came with a variant of 31.1.1.1 which was an engineering build and had the prefix (software update required) on every CLI prompt.    We upgraded to 31.1.1.2 to remove that prompt and then 31.2.1.1 to discover our lower speed devices worked then.

You might like the 30.7 branch, but the 5520’s came from the factory with a higher version and being they are in my opinon, bleeding edge, not fully baked in yet I don’t know if you can go back.  Perhaps EXOS 31 was the first release to support 5520’s?  They had to come with that version and not a more stable prior version for a reason, right?

[Stefan_K_]

Whats the exact SFP type / part number?

But looks like a case for GTAC. Could be another bug in the 31.x firmware. This is why I will stick to the 30.7 branch as long as possible and hopefully won’t get in contact with 5520s too often...