topic RE: Migrate from 1GB to 10GB uplinks - help? in Network Architecture & Design

Migrate from 1GB to 10GB uplinks - help?

Frank — Wed, 08 Mar 2017 18:44:00 GMT

I have a simple setup - two core switches (8800s), several edge switches (460s) connected like this: 8800s are running 16.1.3.6-patch1-9, 460-1 is on 15.4.1.3, 460-2 is on 15.6.1.4.

My problem is that the 460s are on 1GB fiber (ports 55,57) and I need to upgrade them to 10GB while everything stays up and running, i.e. the devices connected to the 460s don't lose access to the network.
The end goal is that the 10GB uplinks also need to be in ports 55,57 (for sanity's sake) Ideally, the corresponding ports in the 8800 should also stay the same (sanity!), but I could be persuaded to use different ports.

If I understand correctly, I can't just replace the SFPs one by one because of port speeds and sharing configs/mismatches.
Physically, the two 460s are right on top of each other, so I could run something between them and use that as an alternate path, but I also hear that loops are deadly. I don't think I can easily define that as a 460-ISC MLAG, as I'd have to put the ports on the 8800 side into sharing mode, affecting vlan-port associations to the 460. (Let's say vlan 1 goes to 460-1 vlan 2 goes to 460-2, let's say on the 8800s port 1:11 goes to 460-1, port 1:12 goes to 460-2. What all would die if I group 1:11 and 1:12? At least all vlans pointing to 1:12 would have an issue, right?)

On the other hand, I'm not currently running spanning-tree. If I connect the 460-1/2 with a gig-ether (or two , shared) in a non-MLAG setup, make sure all vlans that exist on both switches are defined to go to to both switches (and the connection between the 460s) and define spanning-tree on all the vlans, would that get me to a point where I can just rip out one 460's fiber uplink, replace it with 10G, then do the other 460, then disconnect the link between the 460s and kill off STP?

I've never configured STP on Extreme switches, btw. Especially not with share groups and an MLAG.

So, how can I pull this off without downtime (or downtime in the "few seconds" range)? Would STP be viable? Is there something better I can do during the fiber/SFP replacements?

Thanks for your help!
Frank

RE: Migrate from 1GB to 10GB uplinks - help?

Michal_Rz — Wed, 08 Mar 2017 19:12:00 GMT

I would do it like that: configure redundand link (software) on first x460 between shared port and some templorary port to second 460. Make sure all vlans are configured correctly. Then connect them by templorary link, disable and disconnect main links 1g, replace them with 10G and enable both. At the end disconnect templorary links and unconfigure redundant link to clean after work. Next same thing with second 460. No need to make it hard with STP.

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Wed, 08 Mar 2017 19:12:00 GMT

K.I.S.S. I guess I was overthinking things.
Do you know if I do a copy/paste of
enable port 17 (temp.port group)
disable port 55
if there's a delay to establish the new path?

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Wed, 08 Mar 2017 19:12:00 GMT

You could use a script on the switch instead of copy&paste for every configuration line.

Edit: That totally missed the question. 😞

I do not know the fail-over speed for a software redundant link.

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Wed, 08 Mar 2017 19:12:00 GMT

Frank, you need to disable both physical ports of the LAG, not just the master port:
disable port 55,57

RE: Migrate from 1GB to 10GB uplinks - help?

Michal_Rz — Wed, 08 Mar 2017 19:12:00 GMT

Frank, you can create script, witch would disable LAG ports, and shedule it by upm to run on middle of the night if it changes something. Impact might be lower, it depends on your enviroment of course.

RE: Migrate from 1GB to 10GB uplinks - help?

Stephane_Grosj1 — Wed, 08 Mar 2017 19:12:00 GMT

it's always tricking to give failover times, but SRP can be < 1s with link on and it's usually around 2-3s with link off.

Going SRP was the first thing coming to my mind as well.

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Wed, 08 Mar 2017 19:55:00 GMT

Hi Frank,

MLAG and STP do not work well together in EXOS, see e.g. Can I combine MLAG and STP? from GTAC Knowledge. Thus the simple way to have STP block one of the uplinks during migration is NOT supported.

IMHO you should investigate Michal's proposed procedure. Edit: see GTAC Knowledge article How to configure Active/Standby port or LAG feature in EXOS.

Erik

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Wed, 08 Mar 2017 19:55:00 GMT

There is a bad typo in the above post. Seems I cannot edit the post any more. 😞

I wanted to write:

Thus the simple way to have STP block one of the uplinks during migration is NOT supported.

RE: Migrate from 1GB to 10GB uplinks - help?

Drew_C — Wed, 08 Mar 2017 19:55:00 GMT

I gotcha, Erik 

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Wed, 08 Mar 2017 19:55:00 GMT

Thanks, Drew.

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Thu, 09 Mar 2017 16:24:00 GMT

Thanks for the tips on smart/software-controlled redundancy!
I'd like to add (from the 16.1 manual - see, I *do* read those things! - Emphasis mine):

The master port is the only port of a load-sharing group that can be configured as either a primary or redundant port. Also, all ports on the load-sharing group must fail before the software-controlled redundancy is triggered.

The remaning question is:
• To configure the switch for the Smart Redundancy feature, use the following command:
enable smartredundancy port_list

Would that be a port-list of the redundant ports, the primary and redundant ports, or (for lazy people like me), just a list of all ports?

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Thu, 09 Mar 2017 16:24:00 GMT

Nevermind. Earlier during the intro-paragraph, it states:

By default, Smart Redundancy is always enabled. If you enable Smart Redundancy, the switch automatically fails over to the redundant port and returns traffic to the primary port after connectivity is restored on that port.
If you do not want the automatic restoration of the primary link when it becomes active, disable Smart Redundancy.

Gracias, Señor Manual 😉

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Thu, 09 Mar 2017 18:46:00 GMT

So I played with port redundancy. Made sure that all VLANs on switch-1 are also known to switch-2, made sure that all vlans are tagged to the backup link (port 40, which is a group of port 40,41) on both switch-1 and switch-2. At this point, ports 40,41 were disabled on both switches, because otherwise I'd have a bad loop - plus, some VLANs have access ports on both switches for ASA failover pairs and other redundancy things.

Then I thought that the "link on" option for redundancy sounded really nifty! Off I went on switch-1:
enable ports 40,41 (still disabled on switch-2)
configure port 55 redundant 40 link on

Now, as soon as I enabled port 40,41 on switch-2, I see RX traffic on switch-1. Not TX, but RX, so switch-2 decided (rightfully so) to send appropriate traffic to switch-1.
I didn't quite expect switch-1 to receive anything due to the software blocking the redundant link. It appears as if it only blocks TX, but not RX? (15.4.1.3 patch 1-10)
Got a little hectic, disabled the redundancy ports, reconfigured with "Link Off", now things appear to be working as expected.

Is the expected behavior on "link on" to only block TX while the primary port is still up? Is it perhaps because I used a redundant group?

RE: Migrate from 1GB to 10GB uplinks - help?

Michal_Rz — Thu, 09 Mar 2017 18:46:00 GMT

If link redundant is configured to be on on switch-1, then switch-2 will send only
broadcast's, because from his perspective link is working normal, but aint see who is on the other side, thats why It send only broadcast and not unicasts. Also switch-1 can recive EDP, CDP packets or something else like that. To test it simple is to do
show edp port 40,41
and you will see neighbor, so EDP packets must goes through redundant port.

You can check traffic by yourself by capturing traffic on this port.
https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-perform-a-local-packet-capture-on-an-EXOS-switch
I have done it to test my theory 🙂

Also turning redundant link to be off it will extend switching process time from primary to secondary port.

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Thu, 09 Mar 2017 19:21:00 GMT

Hi Frank,

according to How to validate if a software controlled redundant port is set up to block traffic?, TX and RX should be blocked.

Did you verify if the received traffic is actually forwarded to any port or local interface? I'd do that in a lab setting or very carefully...

If you configure redundant ports on both switch-1 and switch-2 before enabling the physical ports, you should see no traffic even with "link on."

Erik

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Thu, 09 Mar 2017 19:21:00 GMT

According to my graphs, something went haywire as heck. Some day when I can make the time , I'll dig deeper into that in a lab setting. For right now, I'm afraid to tick off too many customers if "very carefully" goes sideways.

As to configuring redundancy on both switches, I'm hesitant - if I use the same redundancy port (40) on both sides, I'm not sure what'll happen in case of a failure. If switch-1's port 55 fails, port 40 goes active. However, since switch-2's port 55 is still active, wouldn't it continue to block port 40 / have link-off? Regardless of whether we'd use "link-on" or "link-off"?

RE: Migrate from 1GB to 10GB uplinks - help?

Erik_Auerswald — Thu, 09 Mar 2017 19:21:00 GMT

You are correct, you can use the redundant port on one switch only, as the other does not know about it.

RE: Migrate from 1GB to 10GB uplinks - help?

Michal_Rz — Thu, 09 Mar 2017 19:21:00 GMT

Eric, check my response above

RE: Migrate from 1GB to 10GB uplinks - help?

Frank — Fri, 10 Mar 2017 18:37:00 GMT

Quick update:

I reconfigured the redundancy mode to use "link on", and it did indeed work as advertised! Whatever happened to me yesterday with the traffic/spikes must have been something completely different. Who knows what I did...

During failover from my fiber group to the redundant link group, I may have lost one ping packet, so I'd say 1 second or less to fail is about right.

Re-establishing the link (10G, 2-port share to the MLAG on the 8800s) took about 3-5 seconds.

Thank you very much for all your help, especially Michal for being "first" with the much easier approach for a backup link, and Eric for probably having the entire Q&A section memorized so he could link to all kinds of useful articles!

Have a great weekend!

Frank