I keep finding new ways to get myself into trouble...
I had a two-tier setup like this (Exos 15.4):
Two 480s connected to each other via ISC, ports 1,2 (lag).
Two BDs connected to each other via ISC, ports 1-4 (lag)
Each 480 has one port connected to each of the BDs (lag), which is then MLAGed on the BDs.
In reverse, the ports on the each BD to the 480s are lagged and then MLAGed on the 480s.
From what I understand, that's the way to do this.
A vlan named "Internet" is part of all four switches
All switches have an IP address in vlan "Internet"
Additionally I have VRRP configured on the BDs (prio 250 on BD-1, default prio on BD-2)
create vrrp vlan Internet vrid 1 configure vrrp vlan Internet vrid 1 priority 250
configure vrrp vlan Internet vrid 1 add 10.0.0.254 enable vrrp vlan Internet vrid 1 [/code]I needed to switch the ISC on the 480s from ports 1,2 to ports 11,12 so I created a new share/lag, added that port to vlan "Internet" disconnected the cables in port 1 and 2 on both 480s and reconnected them in ports 11 and 12 on the 480s.
All hell broke loose on the BDs - a flood of notices from BD-1 (none from BD-2):
VRRP.Advert: MSM-A: Advert for VR on vlan Internet vrid 1 ignored: ignoring lower priority advert[/code]
After way too long of trying to figure out what I did wrong, I just disabled ports 11,12 on the 480s and everything returned to normal. Sadly, people who needed to use the Virtual Router IP to get anywhere beyond the "Internet" vlan were, well, effectively shut down
I have no clue what happened. All I can think of is that somehow when I had the 480s configured with the ISC in ports 1,2 something may have blocked things in a way that VRRP broadcasts didn't go wrong/circular/oddly? And that disconnecting/reconnecting in ports 11,12 brought connections up in a way that perhaps caused the VRRP flood?
I'm not even sure if my assumption is anywhere near correct that the BD somehow saw its own VRRP broadcast/advertisement from the wrong port (going in circles?) I've seen some forum posts that make me think that's what's happening and that I should perhaps policy-block VRRP broadcasts on the ports between the BDs and the 480s?
I would need my ISC between the 480s back some day, but I need to make sure that I don't break things in the process.
The good news is that I have another set of switches that I can play with and rebuild my considerably dented confidence.
I guess my questions are:
- did I set up the two-tier config correctly?
- am I even remotely close with my guess about VRRP broadcasts kicking me in the shins?
- How do I fix this mess?
- Eventually I need to also enable VRRP between the 480s. If I'm right with my guess about VRRP broadcasts, I think the "BD-fix" would be analogous to the "480-fix" (policy blocking VRRP broadcasts outside of the 480s / outside of the BDs)
And lastly: why did things not break way sooner
But that could've been mere "luck" on which links were active when (and it's actually more of a curiosity question)
Thanks for your help!