Reauthentication Failure in ExtremeWireless 10.11.01.0210 and NAC 7.0.3.12

[LeoP1]

Hi guys,

I've noticed an odd behavior with a lab deployment of ECC 7.03 and the new EW10.11.01.0210.

After the upgrade of the V2110 to 10.11 the reauth requests by NAC are being ignored (timed out) by the EW.

While using EW 10.01.05.0008 and the same NAC version, everything worked fine. Maybe a 10.11 version bug?

I got the same results while using a 802.1x authenticated VNS and a Open (NAC MAC authenticated for guests) VNS.

Any ideas??

Example 1:

An user registers at NAC Portal (authenticated registration), it is added to the "Web Authenticated Users" and still tied to the Unregistered role (and still looking as Unauthenticated at the EW) when it's expected to move the the Enterprise User role.

The only way to make it work is to Disassociate the user from the EW, then the device reconnects and everything works fine.

Example 2:

A registered device is connected to the network using an EW VNS and if you click the "Force Reauth" button in NAC Manager, nothing happens.

Taking a closer look at the "NAC Appliance Events" tab, it shows an error like "Re-authentication failed. Switch: 10.100.0.250, Port : 103, Port Name : SecureIT-1X, Port Alias: SecureIT-1X, MAC: 7C-E9-D3-D7-7C-0B, Reason: END_SYSTEM_REGISTRATION" (the reasons vary).

Looking at the NAC console, enabling verbose logging for Reauth, it shows:

802.1x:

2016-07-12 13:47:23,906 DEBUG [EndSystemActionRequestHandler] Processing action: (reauthentication) on end system: 7C-E9-D3-D7-7C-0B, IP: 10.100.0.21, user: leonardo, reason: UserSpecified(USER_INITIATED_REAUTH), from appliance: false
2016-07-12 13:47:23,906 DEBUG [EndSystemActionRequestHandler] This NAC engine is the current appliance, so reauth.
2016-07-12 13:47:23,906 DEBUG [EndSystemActionRequestHandler] Reauthing end system: 7C-E9-D3-D7-7C-0B
2016-07-12 13:47:23,906 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Calculating if a re-authentication really needs to be performed for reason: USER_INITIATED_REAUTH.
2016-07-12 13:47:23,906 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 The re-authentication request is being processed because the reauth reason: "USER_INITIATED_REAUTH" is not for a data change.
2016-07-12 13:47:23,907 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Re-authentication running for Switch: 10.100.0.250, Port : 103, Port Name : SecureIT-1X, Port Alias: SecureIT-1X, MAC: 7C-E9-D3-D7-7C-0B, Reason: USER_INITIATED_REAUTH
2016-07-12 13:47:23,907 INFO [ReauthSnmpTask] ESDMAC:D7-7C-0B Executing Reauth for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.100.0.21 for NAS switch 10.100.0.250 switchPort 103 reason: USER_INITIATED_REAUTH all sessions
2016-07-12 13:47:23,907 DEBUG [ReauthSnmpTask] ESDMAC:D7-7C-0B Not using toggle link for session: AUTH_8021X => Rejected: false shouldToggleLinkForRejectedEapTlsOnReauth: true ID: 44532257
2016-07-12 13:47:23,907 INFO [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Starting RFC 3576/5176 Reauthorization for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.100.0.21, reason: USER_INITIATED_REAUTH
2016-07-12 13:47:23,907 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Forcing a disconnect from the RFC 3576/5176 due to reason: USER_INITIATED_REAUTH, forces a re-login.
2016-07-12 13:47:23,907 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Cannot update authorization level because reauthentication reason: USER_INITIATED_REAUTH requires the user to login again.
2016-07-12 13:47:23,908 INFO [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Starting RFC 3576/5176 Reauthentication for MAC: 7C-E9-D3-D7-7C-0B on switch: 10.100.0.250 DAS Port: 3799 with shared secret length: 21
2016-07-12 13:47:23,908 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Transmitting Disconnect-Request to IP: 10.100.0.250 Port: 3799 with Attrs:Calling-Station-Id="7C-E9-D3-D7-7C-0B"Event-Timestamp="1468342043"
2016-07-12 13:47:32,916 ERROR [DisconnectMessageReauthenticationWorker] Reauthentication Failed for to IP: 10.100.0.250 Port: 3799 MAC: 7C-E9-D3-D7-7C-0B due to timeout.
2016-07-12 13:47:32,919 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.100.0.21 Re-authentication failed. Switch: 10.100.0.250, Port : 103, Port Name : SecureIT-1X, Port Alias: SecureIT-1X, MAC: 7C-E9-D3-D7-7C-0B, Reason: USER_INITIATED_REAUTH

MAC:

2016-07-12 13:55:40,365 DEBUG [ReauthService] Reauthenticating 1 by MAC address. 2016-07-12 13:55:40,365 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Calculating if a re-authentication really needs to be performed for reason: END_SYSTEM_REGISTRATION. 2016-07-12 13:55:40,365 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 The re-authentication request is being processed because the reauth reason: "END_SYSTEM_REGISTRATION" is not for a data change. 2016-07-12 13:55:40,365 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Re-authentication running for Switch: 10.100.0.250, Port : 109, Port Name : TESTE_L, Port Alias: TESTE_L, MAC: 7C-E9-D3-D7-7C-0B, Reason: END_SYSTEM_REGISTRATION 2016-07-12 13:55:40,365 INFO [ReauthSnmpTask] ESDMAC:D7-7C-0B Executing Reauth for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.200.0.19 for NAS switch 10.100.0.250 switchPort 109 reason: END_SYSTEM_REGISTRATION all sessions 2016-07-12 13:55:40,365 DEBUG [ReauthSnmpTask] ESDMAC:D7-7C-0B Not using toggle link for session: AUTH_MAC => Rejected: false shouldToggleLinkForRejectedEapTlsOnReauth: true ID: 1066653259 2016-07-12 13:55:40,366 INFO [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Starting RFC 3576/5176 Reauthorization for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.200.0.19, reason: END_SYSTEM_REGISTRATION 2016-07-12 13:55:40,366 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 The end-system with MAC: 7C-E9-D3-D7-7C-0B requires a session-timeout which we are allowing in a CoA for the IdentiFi WLC: 10.100.0.250 with sysObjectId: 1.3.6.1.4.1.4329.15.1.1.13 running firmware version: 10.11.1.210 because it is running firmware greater than: 9.21.1 2016-07-12 13:55:40,366 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 resetAuth for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.200.0.19 on an end-system registration expired, authType = AUTH_MAC_PAP 2016-07-12 13:55:40,377 INFO [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Starting RFC 3576/5176 Reauthorization for MAC: 7C-E9-D3-D7-7C-0B on switch: 10.100.0.250 DAS Port: 3799 with shared secret length: 21 2016-07-12 13:55:40,377 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Transmitting CoA-Request to IP: 10.100.0.250 Port: 3799 with Attrs:Filter-Id="Enterasys:version=1:policy=SecureIT"Calling-Station-Id="7C-E9-D3-D7-7C-0B"Event-Timestamp="1468342540" 2016-07-12 13:55:49,387 ERROR [DisconnectMessageReauthenticationWorker] Reauthentication Failed for to IP: 10.100.0.250 Port: 3799 MAC: 7C-E9-D3-D7-7C-0B due to timeout. 2016-07-12 13:55:49,387 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 The CoA result was *not* applied due to RFC3576 result: FAILED 2016-07-12 13:55:49,387 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Update of authorization level for MAC: 7C-E9-D3-D7-7C-0B, IP: 10.200.0.19 failed, discarding the delayed update: No need to link down the system sending system update 3... 2016-07-12 13:55:49,387 INFO [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Starting RFC 3576/5176 Reauthentication for MAC: 7C-E9-D3-D7-7C-0B on switch: 10.100.0.250 DAS Port: 3799 with shared secret length: 21 2016-07-12 13:55:49,388 DEBUG [DisconnectMessageReauthenticationWorker] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Transmitting Disconnect-Request to IP: 10.100.0.250 Port: 3799 with Attrs:Calling-Station-Id="7C-E9-D3-D7-7C-0B"Event-Timestamp="1468342549" 2016-07-12 13:55:58,398 ERROR [DisconnectMessageReauthenticationWorker] Reauthentication Failed for to IP: 10.100.0.250 Port: 3799 MAC: 7C-E9-D3-D7-7C-0B due to timeout.2016-07-12 13:55:58,399 DEBUG [ReauthTask] ESDMAC:D7-7C-0B,ESDIP:10.200.0.19 Re-authentication failed. Switch: 10.100.0.250, Port : 109, Port Name : TESTE_L, Port Alias: TESTE_L, MAC: 7C-E9-D3-D7-7C-0B, Reason: END_SYSTEM_REGISTRATIONBest regards,

-leo

[LeoP1]

Hi Stephen,

Awesome tip!

My fault not checking on the NAC appliance time sync... Now I've double checked all components to make sure that everyone is synced.

Thank you!!!!

[Stephen_McGuire]

Hello, I'm new to this product line but ran into this just a day or so ago. The problem was the same error message and for me was the NTP settings were off on the wireless controllers and the NAC. NAC was way off. Changed to the same NTP server appears to have corrected the issue. You have to SSH in to change it on the NAC, enforcing the command via netsight won't work.

Not sure if that helps but it's simple to check.

Here's the KB article I found: https://gtacknowledge.extremenetworks.com/articles/Solution/NAC-End-Systems-Hung-in-Captive-Portal

[LeoP1]

Hi Guys,

Doing some downgrades and tests (10.11 -> 10.01.05 -> 10.01.04) and I found that the problem happens in all versions... I haven't tried at 9.xx yet, but maybe a NAC 7.0.3.12 issue...

I'll let you know about any progress...

Best regards,

-leo