Migrate from 1GB to 10GB uplinks - help?

  • 0
  • 2
  • Question
  • Updated 2 years ago
  • Answered
I have a simple setup - two core switches (8800s), several edge switches (460s) connected like this: 8800s are running 16.1.3.6-patch1-9, 460-1 is on 15.4.1.3, 460-2 is on 15.6.1.4.

My problem is that the 460s are on 1GB fiber (ports 55,57) and I need to upgrade them to 10GB while everything stays up and running, i.e. the devices connected to the 460s don't lose access to the network.
The end goal is that the 10GB uplinks also need to be in ports 55,57 (for sanity's sake) Ideally, the corresponding ports in the 8800 should also stay the same (sanity!), but I could be persuaded to use different ports.

If I understand correctly, I can't just replace the SFPs one by one because of port speeds and sharing configs/mismatches.
Physically, the two 460s are right on top of each other, so I could run something between them and use that as an alternate path, but I also hear that loops are deadly. I don't think I can easily define that as a 460-ISC MLAG, as I'd have to put the ports on the 8800 side into sharing mode, affecting vlan-port associations to the 460. (Let's say vlan 1 goes to 460-1 vlan 2 goes to 460-2, let's say on the 8800s port 1:11 goes to 460-1, port 1:12 goes to 460-2. What all would die if I group 1:11 and 1:12? At least all vlans pointing to 1:12 would have an issue, right?)

On the other hand, I'm not currently running spanning-tree. If I connect the 460-1/2 with a gig-ether (or two , shared) in a non-MLAG setup, make sure all vlans that exist on both switches are defined to go to to both switches (and the connection between the 460s) and define spanning-tree on all the vlans, would that get me to a point where I can just rip out one 460's fiber uplink, replace it with 10G, then do the other 460, then disconnect the link between the 460s and kill off STP?

I've never configured STP on Extreme switches, btw. Especially not with share groups and an MLAG.

So, how can I pull this off without downtime (or downtime in the "few seconds" range)? Would STP be viable? Is there something better I can do during the fiber/SFP replacements?

Thanks for your help!
   Frank
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb

Posted 2 years ago

  • 0
  • 2
Photo of Michal Rz

Michal Rz

  • 742 Points 500 badge 2x thumb
I would do it like that: configure redundand link (software) on first x460 between shared port and some templorary port to second 460. Make sure all vlans are configured correctly. Then connect them by templorary link, disable and disconnect main links 1g, replace them with 10G and enable both. At the end disconnect templorary links and unconfigure redundant link to clean after work. Next same thing with second 460. No need to make it hard with STP.
Photo of Frank

Frank

  • 3,724 Points 3k badge 2x thumb
K.I.S.S. I guess I was overthinking things.
Do you know if I do a copy/paste of
  enable port 17 (temp.port group)
  disable port 55
if there's a delay to establish the new path?
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,018 Points 10k badge 2x thumb
You could use a script on the switch instead of copy&paste for every configuration line.

Edit: That totally missed the question. :-(

I do not know the fail-over speed for a software redundant link.
(Edited)
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,018 Points 10k badge 2x thumb
Frank, you need to disable both physical ports of the LAG, not just the master port:
disable port 55,57
Photo of Michal Rz

Michal Rz

  • 742 Points 500 badge 2x thumb
Frank, you can create script, witch would disable LAG ports, and shedule it by upm to run on middle of the night if it changes something. Impact might be lower, it depends on your enviroment of course.
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 13,088 Points 10k badge 2x thumb
it's always tricking to give failover times, but SRP can be < 1s with link on and it's usually around 2-3s with link off.

Going SRP was the first thing coming to my mind as well.
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,458 Points 10k badge 2x thumb
Hi Frank,

MLAG and STP do not work well together in EXOS, see e.g. Can I combine MLAG and STP? from GTAC Knowledge. Thus the simple way to have STP block one of the uplinks during migration is NOT supported.

IMHO you should investigate Michal's proposed procedure. Edit: see GTAC Knowledge article How to configure Active/Standby port or LAG feature in EXOS.

Erik
(Edited)
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,458 Points 10k badge 2x thumb
There is a bad typo in the above post. Seems I cannot edit the post any more. :-(

I wanted to write:
Thus the simple way to have STP block one of the uplinks during migration is NOT supported.
Photo of Drew C.

Drew C., Community Manager

  • 39,442 Points 20k badge 2x thumb
I gotcha, Erik :)
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,458 Points 10k badge 2x thumb
Thanks, Drew.
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb
Thanks for the tips on smart/software-controlled redundancy!
I'd like to add (from the 16.1 manual - see, I *do* read those things! - Emphasis mine):
  • The master port is the only port of a load-sharing group that can be configured as either a primary or redundant port. Also, all ports on the load-sharing group must fail before the software-controlled redundancy is triggered.
The remaning question is:
• To configure the switch for the Smart Redundancy feature, use the following command:
enable smartredundancy port_list

Would that be a port-list of the redundant ports, the primary and redundant ports, or (for lazy people like me), just a list of all ports?
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb
Nevermind. Earlier during the intro-paragraph, it states:

By default, Smart Redundancy is always enabled. If you enable Smart Redundancy, the switch automatically fails over to the redundant port and returns traffic to the primary port after connectivity is restored on that port.
If you do not want the automatic restoration of the primary link when it becomes active, disable Smart Redundancy.

Gracias, Señor Manual ;)
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb
So I played with port redundancy. Made sure that all VLANs  on switch-1 are also known to switch-2, made sure that all vlans are tagged to the backup link (port 40, which is a group of port 40,41) on both switch-1 and switch-2. At this point, ports 40,41 were disabled on both switches, because otherwise I'd have a bad loop - plus, some VLANs have access ports on both switches for ASA failover pairs and other redundancy things.

Then I thought that the "link on" option for redundancy sounded really nifty! Off I went on switch-1:
 enable ports 40,41 (still disabled on switch-2)
 configure port 55 redundant 40 link on

Now, as soon as I enabled port 40,41 on switch-2, I see RX traffic on switch-1. Not TX, but RX, so switch-2 decided (rightfully so) to send appropriate traffic to switch-1.
I didn't quite expect switch-1 to receive anything due to the software blocking the redundant link. It appears as if it only blocks TX, but not RX? (15.4.1.3 patch 1-10)
Got a little hectic, disabled the redundancy ports, reconfigured with "Link Off", now things appear to be working as expected.

Is the expected behavior on "link on" to only block TX while the primary port is still up? Is it perhaps because I used a redundant group?
Photo of Michal Rz

Michal Rz

  • 742 Points 500 badge 2x thumb
If link redundant is configured to be on on switch-1, then switch-2 will send only
broadcast's, because from his perspective link is working normal, but aint see who is on the other side, thats why It send only broadcast and not unicasts. Also switch-1 can recive EDP, CDP packets or something else like that. To test it simple is to do
show edp port 40,41
and you will see neighbor, so EDP packets must goes through redundant port.

You can check traffic by yourself by capturing traffic on this port.
https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-perform-a-local-packet-capture-on-an-EXOS-switch
I have done it to test my theory :)

Also turning redundant link to be off it will extend switching process time from primary to secondary port.
(Edited)
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,458 Points 10k badge 2x thumb
Hi Frank,

according to How to validate if a software controlled redundant port is set up to block traffic?, TX and RX should be blocked.

Did you verify if the received traffic is actually forwarded to any port or local interface? I'd do that in a lab setting or very carefully...

If you configure redundant ports on both switch-1 and switch-2 before enabling the physical ports, you should see no traffic even with "link on."

Erik
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb
According to my graphs, something went haywire as heck. Some day when I can make the time <insert cackles of insanity>, I'll dig deeper into that in a lab setting. For right now, I'm afraid to tick off too many customers if "very carefully" goes sideways.

As to configuring redundancy on both switches, I'm hesitant - if I use the same redundancy port (40) on both sides, I'm not sure what'll happen in case of a failure. If switch-1's port 55 fails, port 40 goes active. However, since switch-2's port 55 is still active, wouldn't it continue to block port 40 / have link-off? Regardless of whether we'd use "link-on" or "link-off"?
Photo of Erik Auerswald

Erik Auerswald, Embassador

  • 13,458 Points 10k badge 2x thumb
You are correct, you can use the redundant port on one switch only, as the other does not know about it.
Photo of Michal Rz

Michal Rz

  • 742 Points 500 badge 2x thumb
Eric, check my response above
Photo of Frank

Frank

  • 3,776 Points 3k badge 2x thumb
Quick update:

I reconfigured the redundancy mode to use "link on", and it did indeed work as advertised! Whatever happened to me yesterday with the traffic/spikes must have been something completely different. Who knows what I did...

During failover from my fiber group to the redundant link group, I may have lost one ping packet, so I'd say 1 second or less to fail is about right.

Re-establishing the link (10G, 2-port share to the MLAG on the 8800s) took about 3-5 seconds.

Thank you very much for all your help, especially Michal for being "first" with the much easier approach for a backup link, and Eric for probably having the entire Q&A section memorized so he could link to all kinds of useful articles!

Have a great weekend!

   Frank