Data Center (SLX)

Expand all | Collapse all

Can't Reach Interface After Changing VRF

  • 1.  Can't Reach Interface After Changing VRF

    Posted 03-16-2019 18:03
    Hi,

    There is probably a very simple answer to this question, but can't think what it is?

    Very high level I have the following configuration:

    code:
    interface vlan 1999
    description "Server Switch Management"
    exit

    config t
    rbridge 11
    interface ve 1999
    ip address 172.24.110.11/24
    ip route 0.0.0.0/0 172.24.110.254

    vcs virtual ip address 172.24.110.10/24 inband interface ve 1999

    configure
    interface port-channel 10
    switchport
    switchport mode trunk
    switchport trunk allowed vlan add 1999
    switchport trunk tag native-vlan
    no shutdown

    int te 11/0/1
    channel-group 10 mode active type standard
    lacp default-up
    int te 12/0/1
    channel-group 10 mode active type standard
    lacp default-up


    This worked find and I could reach both the virtual IP address (172.24.110.10) and the Vlan 1999 interface (172.24.110.10).

    The logical chassis has no other L3 addresses as is primarily just being used at L2.

    The switch was on a version 6 code, and had been trying to upgrade it to version 7 but just kept erroring that it couldn't reach the SCP / FTP server even though I could ping it.

    It seems in version 6 your unable to select the VRF to use when doing the firmware download, so it is defaulting I believe to mgmt-vrf.

    So what I did is add the following command to ve 1999

    code:
    vrf forwarding mgmt-vrf


    Since I did that I lost contact to switch. I did anticipate that could happen although I'm not sure why it did?

    The other end of the portchannel has the IP address 172.24.110.254, and I would have expected to still been able to reach the switch from the local subnet?

    Although the VRF has changed I would expect the VLAN to automatically just reside to the same VRF i.e. just moved from default to management.

    Hence where I am stuck, perhaps I'm missing another command?

    Many thanks in advance.


  • 2.  RE: Can't Reach Interface After Changing VRF

    Posted 03-18-2019 21:07
    Martin,

    I believe the issue is that anytime you add or change a VRF all L3 configuration is removed from an interface.

    Therefore if you were accessing the device using the IP for VE 1999 I would have expected your connection to be terminated as the IP address for this VE should no longer be configured. I would suggest to console into the device and reconfigure the IP address on the VE.

    Example:

    code:
    VDX1# show run rb 1 int ve 1000
    rbridge-id 1
    interface Ve 1000
    ip proxy-arp
    ip address 10.10.10.1/24
    no shutdown
    !
    !
    Static-Lab-SM08_VDX1# conf t
    Entering configuration mode terminal
    Static-Lab-SM08_VDX1(config)# rb 1
    Static-Lab-SM08_VDX1(config-rbridge-id-1)# int ve 1000
    Static-Lab-SM08_VDX1(config-rbridge-Ve-1000)# vrf forwarding mgmt-vrf
    Static-Lab-SM08_VDX1(config-rbridge-Ve-1000)# end
    Static-Lab-SM08_VDX1# show run rb 1 int ve 1000
    rbridge-id 1
    interface Ve 1000
    vrf forwarding mgmt-vrf
    no shutdown


    Also, regarding your SCP/FTP issues. If you can login as your root account you can attempt to manually connect to the FTP server to verify connectivity, username/password, and file path using standard LInux/CLI FTP commands.

    I hope this helps resolve your issue.

    Mike Morey
    Principal Technical Support Engineer


  • 3.  RE: Can't Reach Interface After Changing VRF

    Posted 03-18-2019 22:07
    Hi Mike,

    Thanks for responding. Got some advise from an Extreme Engineer whom said the same thing, which it was.

    Added the IP address back in, but then had another issue with the default gateway. For anyone reading this it made sense to put the route under the vrf, in this case mgmt-vrf, but it would not except the 'ip route' command.

    Turns out it needs to be added another level down under the address-family, see below:

    code:
    vrf mgmt-vrf
    address-family ipv4 unicast
    ip route 0.0.0.0/0 172.24.110.254


    The problem I have now is that I can access SCP / FTP server, which in this case is ExtremeManagement, and see to have the folder structure, that being /root/, so my directory string needs to be /firmware/images



    When trying to upgrade I have tried all the below, none which work?

    code:
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /firmware/nos7.0.2b
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory / file nos7.0.2b
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /firmware/images/nos7.0.2b
    firmware download logical-chassis ftp rbridge-id all coldboot user anonymours password xxxx host x.x.x.x directory /firmware/images/nos7.0.2b
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/nos7.0.2b
    firmware download logical-chassis ftp rbridge-id all coldboot user anonymours password xxxx host x.x.x.x directory /tftpboot/firmware/images/
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory firmware/images/
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory firmware/images/nos7.0.2b


    code:
     Col-xxx-VSP-Sw11# ping x.x.x.x vrf mgmt-vrf
    Type Control-c to abort
    PING x.x.x.x (x.x.x.x): 56 data bytes
    64 bytes from x.x.x.x: icmp_seq=0 ttl=60 time=3.126 ms
    64 bytes from x.x.x.x: icmp_seq=1 ttl=60 time=2.330 ms
    64 bytes from x.x.x.x: icmp_seq=2 ttl=60 time=3.492 ms
    64 bytes from x.x.x.x: icmp_seq=3 ttl=60 time=4.323 ms
    64 bytes from x.x.x.x: icmp_seq=4 ttl=60 time=3.262 ms
    --- x.x.x.xping statistics ---
    5 packets transmitted, 5 packets received, 0% packet loss
    round-trip min/avg/max/stddev = 2.330/3.307/4.323/0.641 ms


    code:
    Col-xxx-VSP-Sw11# show version

    Network Operating System Software
    Network Operating System Version: 6.0.2
    Copyright (c) 1995-2015 Brocade Communications Systems, Inc.
    Firmware name: 6.0.2h
    Build Time: 22:04:55 Apr 12, 2018
    Install Time: 19:35:32 Feb 8, 2019
    Kernel: 2.6.34.6

    BootProm: 1.0.1
    Control Processor: e500mc with 4096 MB of memory

    Slot Name Primary/Secondary Versions Status
    ---------------------------------------------------------------------------
    SW/0 NOS 6.0.2h ACTIVE*
    6.0.2h
    SW/1 NOS 6.0.2h STANDBY
    6.0.2h


    Can you see anything that I am missing or got incorrect?

    Many thanks.


  • 4.  RE: Can't Reach Interface After Changing VRF

    Posted 03-18-2019 23:36
    This looks to be a path issue.

    code:
    firmware download logical-chassis scp rbridge-id all coldboot user root password xxxx host x.x.x.x directory /root/firmware/images/nos7.0.2b


    Another user had similar issues here:

    https://community.extremenetworks.com/data-center-slx-vdx-mlx-ces-232983/upgrading-vdx-over-scp-7822679

    Can you collect the same output from your SSH server and provide it?

    code:
    sw0# ssh x.x.x.x -l root vrf mgmt-vrf


    Once you connected to your SCP server, run the following and paste it back here:

    code:
    $ ls -R /root/firmware/images/nos7.0.2b | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/  /' -e 's/-/|/' | head -20


  • 5.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 07:36
    Hi Truyen,

    Thanks for getting back, here is the results:

    root@NetSightCOL01.abc.co.uk:~$ pwd
    /root

    root@NetSightCOL01.abc.co.uk:~$ ls -R /root/firmware/images/nos7.0.2b | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/' | head -20
    |-----nos7.0.2b
    |-------common
    |---------BP
    |-------SWBD1000
    |-------SWBD1001
    |-------SWBD1002
    |-------SWBD1003
    |-------SWBD1004
    |-------SWBD1005
    |-------SWBD1006
    |-------SWBD1007
    |-------SWBD1008
    |-------SWBD1009
    |-------SWBD1010
    |-------SWBD1011
    |-------SWBD131
    |-------SWBD137
    |-------SWBD138
    |-------SWBD151
    |-------SWBD153

    Wondering if its a permission thing in ExtremeManagement, going to take a look at that next.


    In the past I've had to add the -d to the Netsight nstftpd.cfg file to be able to use TFTP, although it does potentially make it less secure.

    Will perhaps try a different SCP / SFTP server?

    Thanks,

    Martin


  • 6.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 10:57
    Ok, so seems problem was to do with using ExtremeManagement for SCP / SFTP as local FTP worked fine.

    Issue I have now is that I wanted to add an IP address to each of the Rbridges. There is currently 8, and when I try and create interface ve 1999 on another Rbridge I get the following error:

    code:
    Col-xxx-VSP-Sw11(config-rbridge-id-12)# interface Ve 1999
    Generic NSM Backend Error


    Any idea what might be causing that?

    Thanks


  • 7.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 12:44
    As it states, the message is Generic, however what I suspect could be happening is that the VLAN was not correctly provisioned on this RB. You can verify this by issuing

    code:
    show vlan brief


    If this is the case you can try deleting/recreating the VLAN or reloading the box to see if the error persists.


  • 8.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 13:46
    Hi Michael,

    The output of the command you requested is below.

    My understanding (rightly or wrongly) is that I'm running the VDX in logical chassis mode, so from a layer 2 perspective creating the VLAN 1999 should exist on every switch.

    When I going to 'Rbridge 12' it effectively puts me into the layer 3 router configuration, to which I want to create an IP address on each Rbridge for that VLAN, and where I'm hitting the error.

    If on Rbridge 12 I pick a VLAN, say 151, that doesn't have an L3 address configured anywhere this works without a problem. Seems to be just related to ve 1999 that I have an IP address configured on Rbridge 11.

    I'm just in the process of upgrading the switches via USB, as without being able to configure the IP's on the other Rbridge's I couldn't do it over the network.

    Going from 6.02 to 7.0.2b, and then to 7.2.

    The action of upgrading will reboot the switches, so will try again after that and report back.

    code:
    Col-xx-VSP-Sw11# show vlan brief
    Total Number of VLANs configured : 23
    Total Number of VLANs provisioned : 23
    Total Number of VLANs unprovisioned : 0
    VLAN Name State Ports Classification
    (F)-FCoE (u)-Untagged
    (R)-RSPAN (c)-Converged
    (T)-TRANSPARENT (t)-Tagged
    ================ =============== ========================== =============== ====================
    1 default ACTIVE Po 10(t)
    Po 11(t)
    Po 64(t)
    22 VLAN0022 INACTIVE(member port down) Po 11(t)
    30 VLAN0030 INACTIVE(member port down) Po 11(t)
    64 VLAN0064 INACTIVE(member port down) Po 11(t)
    70 VLAN0070 INACTIVE(member port down) Po 11(t)
    71 VLAN0071 INACTIVE(member port down) Po 11(t)
    102 VLAN0102 INACTIVE(member port down) Po 11(t)
    146 VLAN0146 INACTIVE(member port down) Po 11(t)
    147 VLAN0147 INACTIVE(member port down) Po 11(t)
    148 VLAN0148 INACTIVE(member port down) Po 11(t)
    149 VLAN0149 INACTIVE(member port down) Po 11(t)
    150 VLAN0150 INACTIVE(member port down) Po 11(t)
    151 VLAN0151 INACTIVE(member port down) Po 11(t)
    199 VLAN0199 INACTIVE(member port down) Po 11(t)
    240 VLAN0240 INACTIVE(member port down) Po 11(t)
    252 VLAN0252 INACTIVE(member port down) Po 11(t)
    1002(F) VLAN1002 INACTIVE(no member port)
    1164 VLAN1164 ACTIVE Po 10(t)
    Po 64(t)
    1264 VLAN1264 ACTIVE Po 10(t)
    Po 64(t)
    1999 VLAN1999 ACTIVE Po 10(t)
    2002 VLAN2002 INACTIVE(member port down) Po 11(t)
    2003 VLAN2003 INACTIVE(member port down) Po 11(t)
    3333 VLAN3333 INACTIVE(member port down) Po 11(t)


    Many thanks,

    Martin


  • 9.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 13:54
    Martin,

    Your understanding is correct. VLANs are created globally and should be active on all RBs in a VCS fabric. My point is that the NSM backend error is likely caused by the RB not being in sync with this particular VLAN. In order to force this sync, you can remove and re-add the VLAN or reload the offending device so that it performs a config replay.


  • 10.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 14:48
    Hi Michael,

    Your thoughts are correct. The reboot, and / or action of upgrading the switches seems to have corrected the problem and subsequently been able to create an interface ve 1999 on all the remaining Rbridges,

    Thanks to you both for your perseverance.


  • 11.  RE: Can't Reach Interface After Changing VRF

    Posted 03-19-2019 15:19
    Thanks Martin,

    Glad to hear that resolved your issue.

    Michael Morey
    Principal Technical Support Engineer