intersite connectivity - dual fiber, dual cores, dual vendor cores - suggestions?

  • 0
  • 1
  • Question
  • Updated 3 years ago
  • Answered
Here is what I have today:
1. Two sites with interconnected dual core (chassis) switches. Site 1 is Extreme BD8810s. Site 2 is Brocade chassis switches.
2. Both sites running vrrp (for local vlans) and ospf. 
3. Each site has it's own default gateway (local firewall) as well as partner routers (to same partner) at each site redistributing bgp routes into the ospf core. Redundancy for each site works beautifully for the partner sites - each site uses it's local partner router unless it stops putting out routes (goes to other site if it does).
4. Core 1 to Core 1 connection is a 1Gbps LX fiber on a Core 1 to Core 1 only ospf vlan. Core 2 to Core 2 connection is a 100Mbps LX fiber on a Core 2 to Core 2 only ospf vlan. Routes for other site always go through Core 1 to Core 1 link unless the link stops passing traffic in which case it fail over beautifully to the 100Mbps link on Core 2 to Core 2.
5. All attached devices at each site are split between the cores in an active (core 1)/standby (core 2) configuration.

Here is the change:
New Core 2 to Core 2 connection that is 1Gbps LX fiber.

Where I want to go:
I want to actively use both links for 2Gbps between sites.

Questions
1. Will the link speed change and OSPF simply accomplish this in the current configuration? (I do have ospf equal cost sharing enabled)
2. Should I make a single ospf vlan across all 4 switches for this (as opposed to a core1 to core1 vlan and a core2 to core2 vlan)?
3. Extreme side is not running stp but Brocade side is running stp. Will this mess with things?
4. Is there a better way to do this considering I do not want to move away from the current config with local vrrp vlans and ospf?

Redundancy is very important to us as it is a call center environment; I do not want to introduce any single points of failure. I have looked at a couple "smarter" solutions that would let us granularly control traffic between sites but seeing that the connections are fiber (and always up to the demarc even if it is down somewhere in between) a routing or health check type solution is preferred.

Thanks for your advice. I can upload a quickie visio if needed but I think it should be clear from the above.
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb

Posted 3 years ago

  • 0
  • 1
Photo of Mrxlazuardin

Mrxlazuardin

  • 1,534 Points 1k badge 2x thumb
Hi Kevin,

It is hard to understand your topology by sentences. Can you share some topology diagram?

Best regards,
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
will upload one today ... thanks
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
This is an overview diagram. Change will be 100Mbps LX Fiber between Core 2 switches at Site A and Site B will be replaced with 1Gbps LX Fiber. 
Photo of Mrxlazuardin

Mrxlazuardin

  • 1,534 Points 1k badge 2x thumb
Hi Kevin,

1. I don't think your ECMP will work because VRRP will always use main Primary VRRP Router of it is available and the cost to reach other site via backup link is not same with the cost via main link (it must traverse via 10GbE link first).

2. Be careful with L2 loop if you make single VLAN of 4 core switches.

3. I'm not so sure the purpose of STP and for which links STP are implemented.

4. You can try swapping the Primary VRRP Router of some LAN side VLANs so the traffic will go via both links due to different Primary VRRP Router.

Best regards,
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
Thanks! Do you know of any appliance that will allow me to trunk the (2) 1Gbps links and not introduce a single point of failure? Kind of tough with the fiber connections.

In the past we had a pair of x440-8t switches stacked at each site with 100Mbps on stack member 1 and the other 100Mbps on stack member 2. I had the stacks with a port on member 1 connected to core 1 and a sw redundant port for that link on stack member 2 to core 2. Unfortunately LACP sharing did not work very well over the stacking cables. We switched to a static sharing and that worked fine but since the ports are always up to the demarc even when the wide-area portion of the fiber links are down it was scary. We soon replaced one of the fibers with a 1Gbps which is where we are at today. I'll throw up a diagram of the old stack config. I wonder if Extreme ever fixed the issues with LACP across stacking cables? It was HOURS of working with Extreme Support and it was not resolved with us. Since we got the 1Gbps link we never went back to that.
(Edited)
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
old config using x440 stacks at each site to create a sharing for the old (2) 100Mbps fiber links
Photo of Mel78, CISSP, ECE

Mel78, CISSP, ECE

  • 1,044 Points 1k badge 2x thumb

So you removed all EXOS summit X440 from the sites facing each other. And now you have to interoperate BD with Brocade chassis switches. With Site A Core1-2 running EXOS, you can run MLAG. But again, I do not know how the Site 2 Core1-2 running Brocade supports cross chassis LACP (aka MLAG or Cisco term McLAG).

Remember for VoIP sensitive calls (PBX or call center), its always recommended to use L2. For L3 although it is more reliable and offer more granularity QoS (L3 ToS vs L2 CoS) but you will have more latency introduce during heavy traffic.

I assume Site A and Site B and connected by dark fibers. If the sites are link by ISPs or MPLS, then you have no choice but to go L3 for load-sharing.

Please lookup google to know the difference between loadsharing and load-balancing. There is no perfect ratio exists. And the worse case is you have asymmetric routing traffic and this causes chaos and nightmare troubleshooting.


Do no over complicate things.


With your past design X440T, you have wasted a good opportunity. You should have gone with SummitStack-V over the 4 switches using 10G SFP+ or DAC. And this will remove the LACP static/dynamic headache that you have suffered. Assuming your site to site link is dark fiber as SummitStack-V only support point to point connection and uses non-Ethernet protocol.



(Edited)
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
Wong Peng Leong, thank you very much for your reply. I still have the x440-8t switches at each site. I am going to read up on SummitStack-V tomorrow as I am unfamiliar with it. Unfortunately, the ISPs are different for each of the 1Gbps dark fiber connections ... we specifically had them provisioned using different ISPs with completely different paths between sites. Thanks again - Kevin
Photo of Mel78, CISSP, ECE

Mel78, CISSP, ECE

  • 1,044 Points 1k badge 2x thumb

Hi Kevin,


You may look for more info here. Do note the switch model constraints on the usage of 10Gbps for stacking. http://documentation.extremenetworks.com/summit/Summit_Family_HW_Install/Stacking/c_using-the-summit...

Also you may want to use EAPS over SummitStack-V... more configuration for you to manage per switchs, but they both have the same failover timing of <50ms.... totally seamlessly or undetectable for VoiP calls.

Photo of Mrxlazuardin

Mrxlazuardin

  • 1,534 Points 1k badge 2x thumb
Hi Kevin,

I have tried bonding of stacked X460s, but I don't think BD support stacking. At least you can use MLAG which is supported by BD. Both solution, need the same way on Brocade side. If your site to site link is on your own (not leased one from NSP/ISP) you can try stacked X670 for 10Gbps site to site link.

Best regards,
Photo of Kevin Sanders

Kevin Sanders

  • 170 Points 100 badge 2x thumb
Thanks all ... I looked at SummitStack-V and it is only for 10GB ports (our links are 1Gbps from the ISPs and the x440-8t is not supported. I'll take a look at MLAG on both the Extreme and the Brocade.