BGP Edge router suggestions?

[Frank]

I think I may start approaching our 480s' boundaries.

We're a multihomed datacenter, connected to let's say 4 upstream providers
We have two providers each on two 480s.
We receive FULL Internet routes + default from all 4 providers
We receive/advertise both IPv4 AND IPv6.
I also advertise a subset of these routes (severely limited) to our two core 8800 switches - mainly so that outbound traffic which traverses the 8800s has a chance of hitting the right egress router (the aforementioned 480s)

I've already made sure to limit the V4 routes to 500,000 from our neighbor adverts, but once they hit that limit, the tear-down/re-establishment of the BGP neighbor session doesn't help matters.
I already have route compression on, as well as "configure forwarding external-tables l3-only ipv4-and-ipv6" and "configure iproute reserved-entries maximum"

BGP process load can shoot up to 95%+ and in extreme cases, I think it's what makes a router reboot occasionally. Not fun! The only way I found to avoid that was to ditch a few routes from that neighbor to drop them to under-500,000. Still playing with and adjusting NLRI based policies. As I have default routes, I can afford to miss a few "real life" routes, but I'd really rather not.

So I'm thinking that we might need bigger boxes. What are my options?

- I want to have "Full Internet Routes plus default", meaning whatever the current mess is. I think it's over 512K (which is its own can of worms, I know) for V4 - and V6, but that's a lot fewer routes!
- I want to be able to support more than one upstream provider per box.
- My two boxes need to be able to talk to each other and exchange BGP routes properly
- If one or two upstream provider die unexpectedly and BGP routes gets seriously reshuffled, I don't want the box to fail.

Does Extreme have a bigger box for that? Or, if not, what would you suggest to get?

Thank you much for your input.

[Stephane_Grosj1]

Hi,

What EXOS release are you using?

With IP Route compression, the actual 540,000 routes are compressed to 280,000, but that's for the FIB. You can see it if you do a "sh iproute summary". IPv6 yields for ~20k more routes, and compress will reduce it significantly as well if you have it turned on (that's a different command than for IPv4).

If correctly set, you shouldn't have an issue with the FIB. Maybe the issue is more with the RIB, and thus the question about EXOS version, maybe you're running a non-optimal version in regards to BGP.

As Kevin pointed out, BDX8 has several blades with XL capacity, offering up to 1M entries in the FIB. The 40G-XL blade can act as a 10G blade as well through breakout cabling.

[Zelnosky__Kevin]

Also, looks like we recently released a 40G12X-XL blade for the BD X8 that is not on the release notes yet. I'm not positive, but it should have the same table size as the 100G4X-XL.

[Frank]

Mostly being able to hold all routes and being stable are my main goals. At this point I'm just trying to figure out what the entry point for such a router might be. "I'm a geek, not a budgeteer, Jim!" 🙂

[Zelnosky__Kevin]

Frank,

According to the 15.7 release notes supported limits, the only thing with bigger table sizes is the BlackDiamond X8 series with BlackDiamond X8-100G4X-XL modules.

The theoretical limit on these looks to be about 1,048,544 based on forwarding external table configuration "l3-only ipv4".

[andreas1]

I do not think extreme has a box that big today. At least that do perform well. Are you looking for fast route updates and stability? What is your budget ?