CRCs on ISL port

  • 0
  • 1
  • Question
  • Updated 3 days ago
  • Answered
Hello,

A fabric of six VDXs 6740T-1G running NOS 6.0.2b.

We started observing CRCs and receive errors on one of the ISL links.

Receive Statistics:
    60102804042 packets, 51259679161172 bytes
    Unicasts: 59225527043, Multicasts: 874747487, Broadcasts: 0
    64-byte pkts: 381, Over 64-byte pkts: 18462668286, Over 127-byte pkts: 6404289237
    Over 255-byte pkts: 3351627659, Over 511-byte pkts: 2901158210, Over 1023-byte pkts: 1241512515
    Over 1518-byte pkts(Jumbo): 27741541072
    Runts: 0, Jabbers: 0, CRC: 2373642, Overruns: 0
    Errors: 162197, Discards: 21813, TrillportCtrlFrames: 53671211
The fiber carrier (since CRCs are present) normally asked us to replace the adapter and check the cable, but the link was CRCs free for quite some time, so I was wondering is there anything else we can try from the switch side?

The ISL port is connected to the 40G slot (split into 4x10G) via Mellanox adapter. The SFP+ adapter is a Brocade one:
  Interface      TenGigabitEthernet 79/0/52:1
  Identifier     3    SFP
  Connector      7    LC
  Transceiver            0000000000000010 10_GB/s  
  Name           sw
  Encoding       6    
  Baud Rate      103 (units 100 megabaud)
  Length 9u      0   (units km)
  Length 9u      0   (units 100 meters)
  Length 50u     8   (units 10 meters)
  Length 62.5u   3   (units 10 meters)
  Length Cu      0   (units 1 meter)
  Vendor Name    BROCADE         
  Vendor OUI     00:05:1e
  Vendor PN      57-0000075-01   
  Vendor Rev     A
  Wavelength     850 (units nm)
  Options        001a 
  BR Max         0   
  BR Min         0   
  Serial No      XXXXXXXXXXXXX
  Date Code      170529  
  Optical Monitor yes
  Temperature    31 Centigrade
  Voltage        3277.2  (mVolts)
  Current        8.722  (mAmps)
  TX Power       595.6  (uWatts)
  RX Power       556.3  (uWatts)
Many thanks in advance.
Photo of kaba1ahxxx .

kaba1ahxxx .

  • 112 Points 100 badge 2x thumb

Posted 1 week ago

  • 0
  • 1
Photo of EtherMAN

EtherMAN, Embassador

  • 7,200 Points 5k badge 2x thumb
CRC are always a layer one issue... Optics do go bad some time on the receive side. 

Most of the time it is a degradation of the light.  You said fiber carrier so are these all in different location and you have a 3rd party providing you fiber for the ISL links?.. The optic you listed seems to be a MM SR which would only be a local link? 

From my experience CRC are almost always fiber related.  Patches got dirty.  Bend or kink in patch. PAtch is not seating in frame or optic correctly.

Out of a several thousands of optic transceivers we have only replaced a hand full where they just go out or start taking CRC's ... We have burnt out a few ER and ZR optics through the years by over saturating the led with an OTDR but that is a human error that can be prevented.  

I did find a previous GTAC https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-troubleshoot-CRC-errors-or-general-...
Photo of kaba1ahxxx .

kaba1ahxxx .

  • 112 Points 100 badge 2x thumb
Thank you!
That's correct: there is a 3rd party providing 10G fiber between our data centers with the distance ~5km. From our switches it terminates directly to their OTN boxes (Huawei OptiX OSN 1800), so the requirement from their end was a 10G MM SR.
They also claim that they see CRCs on both ends, but we only see CRCs on one of the VDXs, the other side is CRC free (at least 'sh int' shows 0 CRCs).
Photo of EtherMAN

EtherMAN, Embassador

  • 7,200 Points 5k badge 2x thumb
Have you got optical readings from your links and theirs?? That would be the place to start.  I would suggest cleaning/replacing patches first and if that does not clear it up then optics.  I would guess patches since you both see CRC's .. I would double check the bend radius along the path and especially at the connectors to make sure that the weight over time has not bent the patch at the connector causing the problem.  Is the mm patch ten gig rated comes to mind too...