Fabric Attach pros/cons

[Anonymous]

Hi all,

I'm halfway through my first Fabric Connect deployment, and looking for in the field experience with Fabric Attach.
At this network's core are two server rooms, with a VSP 7400 and a 5520-24t running VOSS in each one, then we have X450G2 stacks for distribution, along with a few old B5 stacks.
Software versions are VOSS 8.4.2.0 and EXOS 30.7.2.1.

So, Fabric Attach.
Automatically pulling VLANs from the edge switches sounded great initially.

That is, until I realized that:
- FA is dog slow, VLANs get mapped about 1 min after you link comes up, with default LLDP timings
- FA requires full trust in your edge switches, as any I-SID/VLAN in your core is "attachable", and apparently can't be protected from it
- FA cannot handle multiple paths, but relies on LACP for redundancy, so I still need to set this up on both core and edge
- and I still have to deal with port-based VLANs on VOSS anyway, for B5 stacks, servers, routers, etc

Lowering the LLDP transmit interval from 30s to 10s only made FA barely usable, mapping VLANs still takes from 15s to 20s.
I consider using a smaller LLDP interval to be unreasonable, as it's a global value for all ports in the EXOS stack, I and don't want this to affect other devices in any way.
Didn't find any KB article dealing with this issue.
I guess the problem is LLDP advertisements are sent in sync globally to all ports, and we end-up waiting for the next scheduled dispatch, and that's what makes it so painfully slow, but really FA on the edge should trigger a "gratuitous" LLDP advertisement when the link comes up and be done with it.
So long for the Fabric Connect super fast convergence time...

Then, the lack of any core-side control over which VLAN/I-SID can be mapped really annoys me.
We do use FA authentication (no one should use unauth'ed FA in its current state IMO), but that's not enough.
I don't want the edge switches to be able to extend the server or infrastructure VLANs, it just doesn't feel safe.
I can see the value in only provisioning edge VLANs in only one place, but I want to be able to selectively enable VLANs/I-SIDs for FA on the core/server side.

And ultimately, I don't feel that I'm really gaining that much from FA in return.
I still have to configure core to edge ports for MLT/LACP (maybe even switch to trunk mode manually, I don't remember).
I still need to map I-SIDs to actual VLANs on the 7400s in order to provide IP subnets gateways, routing and so on (so I end-up with a double I-SID to VLAN mapping).

Did I miss something here?
Does FA bring any other benefit?
Is there a well hidden trick to make VLAN mappings happen faster?
Is there a way to whitelist I-SIDs for FA? (and prevent any other I-SID from being ever mapped)
Does FA really only make sense in a "real" larger Fabric Connect network?
Am I wrong in considering dropping it in favor of static VLANs?

Thanks,

Nicolas

[Evert]

Hello Roger,
That's good news, thank you!

[Anonymous]

Roger,

Thanks for the good news!
I'm really looking forward to this triggered LLDP updates feature.
A global "fa allowed-vlans" command (only allow specified VLANs to be pulled by FA) is also high on my Xmas wish list. 😄

[EXTR_Paul]

I would consider upgrading to your EXOS switch to 31.5 and see if you can replicate the issue. 
Also are you VSPs core devices in vIST clusters?

to answer some of your questions. 
- FA is dog slow.  
FA is not a resiliency protocol. Its an automation/ztp feature. FA needs to go through a ton of order of operations to make sure that new switch get adopted proper.  If you want to speed it up, make sure Spanning Tree is disabled on your VSP ports.   FA won't start its Mojo until spanning tree is done its business.  I usually see the FA assignments kick in after 10-20 seconds.


- FA requires full trust in your edge switches, as any I-SID/VLAN in your core is "attachable", and apparently can't be protected from it.

FA gives the user granularity in the moves/add/changes in vlans in the edge.  The VLAN needs to exist on the edge switch and the VLAN needs an explicit i-sid. This can be done manually or with XIQ-SE/Control. This is full control.    If you think about how some vendors do tagging.  Some automatically put all VLANs on all trunks and one needs to explicitly prune the vlans away.  Or protocols like VTP that just send VLANs everywhere. 

FA also has security with authentication so only trusted edge switches can receive the FA assignments and elements 

- FA cannot handle multiple paths, but relies on LACP for redundancy, so I still need to set this up on both core and edge.

Not true.  You can manually create a LAG group on the EXOS switch.   Also, with EXOS 31.X there is an auto-lag feature that will sense that the two or more links are connected to an MLT/SMLT from the VSP and will automatically create the lag group on the EXOS switch.  I have tested this a lot. It actually works very fast.  But you need to be running the newest code.

- and I still have to deal with port-based VLANs on VOSS anyway, for B5 stacks, servers, routers, etc.

Not necessarily.  FA will auto create VLANs in the core.  
But usually network admins want that control.

[Anonymous]

Hello Paul,

No, I did not try running 31.x yet, and I'll stay on 30.7 for now, but will definitely give a try as time permits.

> If you want to speed it up, make sure Spanning Tree is disabled on your VSP ports.
But I do want STP! 🙂 I actually got hit hard last week by the lack of STP support for "multihomed" devices in the default VSP configuration (https://extremeportal.force.com/ExtrArticleDetail?an=000082836)
The 30-60 seconds delay is clearly not due to STP though, ports are up in forwarding state loooong before VLANs get mapped by FA.
Also have a deep hate for Cisco's "help me I'm stuck in the 90s" VLANs, but TBH, VOSS makes it difficult and painful as well, especially compared to EOS/EXOS.

Yes, I get your arguments for FA.
I still believe you should not be allowed to pull just any VLAN in your core network from an edge device.

Thanks for telling about the automatic LAG creation in EXOS 31.7 as Mig did, and glad it worked for you.
I need to look into this.

> > and I still have to deal with port-based VLANs on VOSS anyway, for B5 stacks, servers, routers, etc.
> Not necessarily. FA will auto create VLANs in the core.
Unless I missed something, FA won't help with non FA-enabled devices.

[Miguel-Angel_RO]

Hi Nicolas,

A lot of questions there.
Let's answer the main one: "Am I wrong in considering dropping it in favor of static VLANs?"
It depends of you business requirements. For some environments, the FA process is indeed seen as too slow while on others it is ok.
By experience, the standard timers are ok for most of the customers.

Here few comments:
I would urgently upgrade to 8.4.2.1, the 8.4.2.0 has an ugly bug and has been removed from the support site.

On the last EXOS releases you also have the "Fabric Attach Automatic LAG Creation". This automates the LAG creation and VLANs mapping between EXOS and SMLT clusters.
I remember one hospital who cannot cope with the base timers and went  for static LAG configuration.

You can specify a white list of i-sid's allowed via radius authentication, see here : https://extremeportal.force.com/ExtrArticleDetail?an=000073262

On the edge switches you can have a minimalistic config with netlogin on all ports.
The radius will then return the couple VLAN/I-SID (or a role) on an authentication based approach for the clients.

With those features you have a full automated port configuration.
This is not solving the lag time to have the VLAN provisioning but this is happening only for the first client. The second client on the same VLAN will just get the VLAN on the port activated, the uplink will be already provisioned.

I hope this helps,
Mig