topic Can't Reach Interface After Changing VRF in Data Center (SLX)

Can't Reach Interface After Changing VRF

Anonymous — Sun, 17 Mar 2019 01:03:00 GMT

Hi,

There is probably a very simple answer to this question, but can't think what it is?

Very high level I have the following configuration:

code:

interface vlan 1999
description "Server Switch Management"
exit

config t
rbridge 11
interface ve 1999
ip address 172.24.110.11/24
ip route 0.0.0.0/0 172.24.110.254

vcs virtual ip address 172.24.110.10/24 inband interface ve 1999

configure
interface port-channel 10
switchport
switchport mode trunk
switchport trunk allowed vlan add 1999
switchport trunk tag native-vlan
no shutdown

int te 11/0/1
channel-group 10 mode active type standard
lacp default-up
int te 12/0/1
channel-group 10 mode active type standard
lacp default-up

This worked find and I could reach both the virtual IP address (172.24.110.10) and the Vlan 1999 interface (172.24.110.10).

The logical chassis has no other L3 addresses as is primarily just being used at L2.

The switch was on a version 6 code, and had been trying to upgrade it to version 7 but just kept erroring that it couldn't reach the SCP / FTP server even though I could ping it.

It seems in version 6 your unable to select the VRF to use when doing the firmware download, so it is defaulting I believe to mgmt-vrf.

So what I did is add the following command to ve 1999

code:

vrf forwarding mgmt-vrf

Since I did that I lost contact to switch. I did anticipate that could happen although I'm not sure why it did?

The other end of the portchannel has the IP address 172.24.110.254, and I would have expected to still been able to reach the switch from the local subnet?

Although the VRF has changed I would expect the VLAN to automatically just reside to the same VRF i.e. just moved from default to management.

Hence where I am stuck, perhaps I'm missing another command?

Many thanks in advance.

RE: Can't Reach Interface After Changing VRF

Michael_Morey — Tue, 19 Mar 2019 04:07:00 GMT

Martin,

I believe the issue is that anytime you add or change a VRF all L3 configuration is removed from an interface.

Therefore if you were accessing the device using the IP for VE 1999 I would have expected your connection to be terminated as the IP address for this VE should no longer be configured. I would suggest to console into the device and reconfigure the IP address on the VE.

Example:

code:

VDX1# show run rb 1 int ve 1000
rbridge-id 1
 interface Ve 1000
  ip proxy-arp
  ip address 10.10.10.1/24
  no shutdown
 !
!
Static-Lab-SM08_VDX1# conf t
Entering configuration mode terminal
Static-Lab-SM08_VDX1(config)# rb 1
Static-Lab-SM08_VDX1(config-rbridge-id-1)# int ve 1000
Static-Lab-SM08_VDX1(config-rbridge-Ve-1000)# vrf forwarding mgmt-vrf
Static-Lab-SM08_VDX1(config-rbridge-Ve-1000)# end
Static-Lab-SM08_VDX1# show run rb 1 int ve 1000   
rbridge-id 1
 interface Ve 1000
  vrf forwarding mgmt-vrf
  no shutdown

Also, regarding your SCP/FTP issues. If you can login as your root account you can attempt to manually connect to the FTP server to verify connectivity, username/password, and file path using standard LInux/CLI FTP commands.

I hope this helps resolve your issue.

Mike Morey
Principal Technical Support Engineer

RE: Can't Reach Interface After Changing VRF

Anonymous — Tue, 19 Mar 2019 05:07:00 GMT

Hi Mike,

Thanks for responding. Got some advise from an Extreme Engineer whom said the same thing, which it was.

Added the IP address back in, but then had another issue with the default gateway. For anyone reading this it made sense to put the route under the vrf, in this case mgmt-vrf, but it would not except the 'ip route' command.

Turns out it needs to be added another level down under the address-family, see below:

code:

vrf mgmt-vrf
 address-family ipv4 unicast
    ip route 0.0.0.0/0 172.24.110.254

The problem I have now is that I can access SCP / FTP server, which in this case is ExtremeManagement, and see to have the folder structure, that being /root/, so my directory string needs to be /firmware/images

When trying to upgrade I have tried all the below, none which work?

code:

firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /firmware/nos7.0.2b
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory / file nos7.0.2b
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /firmware/images/nos7.0.2b
firmware download logical-chassis ftp rbridge-id all coldboot user anonymours password xxxx host x.x.x.x directory /firmware/images/nos7.0.2b
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/nos7.0.2b
firmware download logical-chassis ftp rbridge-id all coldboot user anonymours password xxxx host x.x.x.x directory /tftpboot/firmware/images/
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory firmware/images/
firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory firmware/images/nos7.0.2b

code:

 Col-xxx-VSP-Sw11# ping x.x.x.x vrf mgmt-vrf
Type Control-c to abort
PING x.x.x.x (x.x.x.x): 56 data bytes
64 bytes from x.x.x.x: icmp_seq=0 ttl=60 time=3.126 ms
64 bytes from x.x.x.x: icmp_seq=1 ttl=60 time=2.330 ms
64 bytes from x.x.x.x: icmp_seq=2 ttl=60 time=3.492 ms
64 bytes from x.x.x.x: icmp_seq=3 ttl=60 time=4.323 ms
64 bytes from x.x.x.x: icmp_seq=4 ttl=60 time=3.262 ms
--- x.x.x.xping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 2.330/3.307/4.323/0.641 ms

code:

Col-xxx-VSP-Sw11# show version

Network Operating System Software
Network Operating System Version: 6.0.2
Copyright (c) 1995-2015 Brocade Communications Systems, Inc.
Firmware name:   6.0.2h
Build Time:     22:04:55 Apr 12, 2018
Install Time:    19:35:32 Feb 8, 2019
Kernel:       2.6.34.6

BootProm:      1.0.1
Control Processor: e500mc with 4096 MB of memory

Slot  Name  Primary/Secondary Versions             Status
---------------------------------------------------------------------------
SW/0  NOS   6.0.2h                       ACTIVE*
        6.0.2h
SW/1  NOS   6.0.2h                       STANDBY
        6.0.2h

Can you see anything that I am missing or got incorrect?

Many thanks.

RE: Can't Reach Interface After Changing VRF

Truyen_Phan — Tue, 19 Mar 2019 06:36:00 GMT

This looks to be a path issue.

code:

firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/nos7.0.2b

Another user had similar issues here:

https://community.extremenetworks.com/data-center-slx-vdx-mlx-ces-232983/upgrading-vdx-over-scp-7822679

Can you collect the same output from your SSH server and provide it?

code:

sw0# ssh x.x.x.x -l root vrf mgmt-vrf

Once you connected to your SCP server, run the following and paste it back here:

code:

$ ls -R /root/firmware/images/nos7.0.2b | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/  /' -e 's/-/|/' | head -20

RE: Can't Reach Interface After Changing VRF

Anonymous — Tue, 19 Mar 2019 14:36:00 GMT

In the past I've had to add the -d to the Netsight nstftpd.cfg file to be able to use TFTP, although it does potentially make it less secure.

Will perhaps try a different SCP / SFTP server?

Thanks,

Martin

RE: Can't Reach Interface After Changing VRF

Anonymous — Tue, 19 Mar 2019 17:57:00 GMT

Ok, so seems problem was to do with using ExtremeManagement for SCP / SFTP as local FTP worked fine.

Issue I have now is that I wanted to add an IP address to each of the Rbridges. There is currently 8, and when I try and create interface ve 1999 on another Rbridge I get the following error:

code:

Col-xxx-VSP-Sw11(config-rbridge-id-12)# interface Ve 1999
Generic NSM Backend Error

Any idea what might be causing that?

Thanks

RE: Can't Reach Interface After Changing VRF

Michael_Morey — Tue, 19 Mar 2019 19:44:00 GMT

As it states, the message is Generic, however what I suspect could be happening is that the VLAN was not correctly provisioned on this RB. You can verify this by issuing

code:

show vlan brief

If this is the case you can try deleting/recreating the VLAN or reloading the box to see if the error persists.

RE: Can't Reach Interface After Changing VRF

Anonymous — Tue, 19 Mar 2019 20:46:00 GMT

Hi Michael,

The output of the command you requested is below.

My understanding (rightly or wrongly) is that I'm running the VDX in logical chassis mode, so from a layer 2 perspective creating the VLAN 1999 should exist on every switch.

When I going to 'Rbridge 12' it effectively puts me into the layer 3 router configuration, to which I want to create an IP address on each Rbridge for that VLAN, and where I'm hitting the error.

If on Rbridge 12 I pick a VLAN, say 151, that doesn't have an L3 address configured anywhere this works without a problem. Seems to be just related to ve 1999 that I have an IP address configured on Rbridge 11.

I'm just in the process of upgrading the switches via USB, as without being able to configure the IP's on the other Rbridge's I couldn't do it over the network.

Going from 6.02 to 7.0.2b, and then to 7.2.

The action of upgrading will reboot the switches, so will try again after that and report back.

code:

Col-xx-VSP-Sw11# show vlan brief
Total Number of VLANs configured   : 23
Total Number of VLANs provisioned  : 23
Total Number of VLANs unprovisioned : 0
VLAN            Name           State                     Ports          Classification
(F)-FCoE                                                   (u)-Untagged
(R)-RSPAN                                                  (c)-Converged
(T)-TRANSPARENT                                            (t)-Tagged
================ =============== ========================== =============== ====================
1               default        ACTIVE                    Po 10(t)
                                                           Po 11(t)
                                                           Po 64(t)
22              VLAN0022       INACTIVE(member port down) Po 11(t)
30              VLAN0030       INACTIVE(member port down) Po 11(t)
64              VLAN0064       INACTIVE(member port down) Po 11(t)
70              VLAN0070       INACTIVE(member port down) Po 11(t)
71              VLAN0071       INACTIVE(member port down) Po 11(t)
102             VLAN0102       INACTIVE(member port down) Po 11(t)
146             VLAN0146       INACTIVE(member port down) Po 11(t)
147             VLAN0147       INACTIVE(member port down) Po 11(t)
148             VLAN0148       INACTIVE(member port down) Po 11(t)
149             VLAN0149       INACTIVE(member port down) Po 11(t)
150             VLAN0150       INACTIVE(member port down) Po 11(t)
151             VLAN0151       INACTIVE(member port down) Po 11(t)
199             VLAN0199       INACTIVE(member port down) Po 11(t)
240             VLAN0240       INACTIVE(member port down) Po 11(t)
252             VLAN0252       INACTIVE(member port down) Po 11(t)
1002(F)         VLAN1002       INACTIVE(no member port)
1164            VLAN1164       ACTIVE                    Po 10(t)
                                                           Po 64(t)
1264            VLAN1264       ACTIVE                    Po 10(t)
                                                           Po 64(t)
1999            VLAN1999       ACTIVE                    Po 10(t)
2002            VLAN2002       INACTIVE(member port down) Po 11(t)
2003            VLAN2003       INACTIVE(member port down) Po 11(t)
3333            VLAN3333       INACTIVE(member port down) Po 11(t)

Many thanks,

Martin

RE: Can't Reach Interface After Changing VRF

Michael_Morey — Tue, 19 Mar 2019 20:54:00 GMT

Martin,

Your understanding is correct. VLANs are created globally and should be active on all RBs in a VCS fabric. My point is that the NSM backend error is likely caused by the RB not being in sync with this particular VLAN. In order to force this sync, you can remove and re-add the VLAN or reload the offending device so that it performs a config replay.

RE: Can't Reach Interface After Changing VRF

Anonymous — Tue, 19 Mar 2019 21:48:00 GMT

Hi Michael,

Your thoughts are correct. The reboot, and / or action of upgrading the switches seems to have corrected the problem and subsequently been able to create an interface ve 1999 on all the remaining Rbridges,

Thanks to you both for your perseverance.

RE: Can't Reach Interface After Changing VRF

Michael_Morey — Tue, 19 Mar 2019 22:19:00 GMT

Thanks Martin,

Glad to hear that resolved your issue.

Michael Morey
Principal Technical Support Engineer