Bypassing link aggregation for a specific MAC address

[EtherNation_Use]

Create Date: Jan 13 2012 9:25AM

I have some Apple Xserves connected to a Summit X460, with their two internal ethernet ports in a LACP LAG. Xserves also have a IPMI based lights-out-management (LOM) device which shares the same physical ethernet ports, but has a distinct MAC address and IP configuration.

The issue is that the lights-out-management device knows nothing about LACP or the link aggregation. Apple officially says that link aggregation and LOM are mutually exclusive, but I think this is an oversimplification. I believe what I need is a way to bypass the link aggregation for traffic to the LOM so that none of it ends up on the wrong port. I tried doing this with an ACL:

entry lom { if { destination-address 172.20.20.207/32; } then { permit; redirect-port-no-sharing 1:21; } } That almost works. I found that other hosts could only reach that IP if they happened to see a gratuitous ARP request from the LOM or if I added a static entry to their ARP table. Is this because the broadcast ARP query looking for who has the LOM's IP is going to the wrong physical port? I configured proxy ARP on the switch, and it seems like most hosts can ping the LOM now, but notably the switch can't. I see the LOM sending replies:

00:19:e3:e7:a9:e8 > cc:5e:09:00:fc:5e, IPv4, length 50: 172.20.20.207 > 172.20.20.1: ICMP echo reply, id 38, seq 3, length 16 But to cc:5e:09:00:fc:5e? What's that? It's not my switch, for sure. I'm confused and stuck. Can anyone provide a hint?

(from Phil_Frost)

[EtherNation_Use]

Create Date: Jan 19 2012 11:25AM

Yes, I think that's the problem, though I'm not entirely sure. The trouble is I can't think of a way to direct the switch to send L2 broadcast traffic to a specified port independently of the LAG settings without also interfering with broadcast frame delivery to the other 46 ports on the switch.

I also realized that by configuring proxy ARP on the switch, I might have simply traded one problem for another. While having the switch reply to ARP queries on behalf of the LOM allows other hosts on the network to resolve the LOM's IP, I think it also might make the LOM think its IP is in use, since when it sends out a gratuitous ARP request, the switch replies. It's rather hard to know exactly what's going on from the LOM's perspective since there are very limited facilities for inspection. Of course if I could get L2 broadcasts to always arrive at the LOM's port, then I wouldn't need proxy ARP. (from Phil_Frost)

[EtherNation_Use]

Create Date: Jan 17 2012 6:28AM

On second thought. You may want to also filter on the broadcast mac address for it to work.

Best regards,

Kenneth

(from Kenneth_Oestrup)

[EtherNation_Use]

Create Date: Jan 17 2012 5:30AM

Hi!

This is a very interresting issue.

As I understand, the problem is that the other devices on the network are unable to send ARP requests to the LOM.

Your ACL only matches on IP address. I think, that if you matched on the MAC address of the LOM it should work.

The LOM should have a seperate MAC address, at least that's how it works on Sun servers.

Please let me know how it turns out 🙂

Best regards,
Kenneth

(from Kenneth_Oestrup)