Tacacs Login Problem

  • 0
  • 1
  • Problem
  • Updated 2 years ago
  • Solved
  • (Edited)
Hello Folks...

I put myself on a big trouble, and I hope you guys can help to get me out of it.

Here in a company I'm working for, we have a Enterasys S8 Chassis with two fabric switches and two blades with normal switches.

We were trying to configure tacacs on it, we have done a lot of tests and we've got no success.

This is the config I used the first time we've tried:

set tacacs server  1 <My IP> 49 <mysecret>
set tacacs command accounting enable
set tacacs command authorization enable
set tacacs enable

This one isn't work. So I removed.

Obs:
I had established a ssh connection to put this configuration, and keep this same ssh window connection to apply or remove this configuration any time I wanted, without any kind problem, and with another ssh windows we tried to connect with our tacacs users.
So, once I still have this ssh connection established, I was able to put another configuration to try get it working, then I put this one:

set tacacs server  1 <My IP> 49 <mysecret>
set tacacs command accounting enable
set tacacs command authorization enable
set tacacs enable

Exact the same, but at the end of it, I added:

set authentication login tacacs

And nothing, it haven't worked as well. So we decided to stop the tests for a while and try to make it work another day. Then we removed all the configuration above, but we forgot to remove the authentication line (set authentication login tacacs).

I read that this configuration turns the tacacs as my primary login method.

We've closed all of ssh connection windows, and since that moment I've logged out from my switch, I wasn't able to login on it anymore, neither with my tacacs user nor with my local user.

I don't know what to do.

Is there some way to login in this switch on a recovery mode, or boot it skipping my current configuration, so this way I would be able to change the configuration on my switch??

Please, someone, help me.

Sorry for the weak English and the Big Text.
Photo of Michel Braga Guimaraes

Michel Braga Guimaraes

  • 194 Points 100 badge 2x thumb
  • frustraded

Posted 3 years ago

  • 0
  • 1
Photo of Dorian Perry

Dorian Perry, Employee

  • 2,300 Points 2k badge 2x thumb
Hi Michel,

Are you using NetSight to manage this switch?
Not yet, my intention was to do that, but unfortunately I'm locked out.

I was using the pure ssh.
Photo of Curtis Parish

Curtis Parish

  • 3,332 Points 3k badge 2x thumb
Do you know the read/write SNMP credentials of the switch?   (default or you set them?)  If so you can still add the switch to netsight.  I have recovered locked out switches using Netsight.   There is a method of resetting the boards that involves removing them and flipping some DIP switches.  Not sure if that method is published or if GTAC will have to walk you through it.       Netsight can also copy the config off before the reset if you can get it set up. 
(Edited)
I think the switch have the snmp configuration on it's default.

Do you know the name of the default read-write community?

Since now I appreciate your help.
Photo of Drew C.

Drew C., Community Manager

  • 38,276 Points 20k badge 2x thumb
Hi Michel, were you able to get back into the switch?
Unfortunately no... Still trying to find a way to login on it. Do you know somehow??
Photo of Drew C.

Drew C., Community Manager

  • 38,276 Points 20k badge 2x thumb
I don't know for sure, but the defaults are usually public / private for read and write communities.
Photo of Matthew Hum

Matthew Hum, Principal Engineer, APAC

  • 1,542 Points 1k badge 2x thumb
If you remove the uplink, and the switch gets no response back from the TACACS server I think it defaults to local accounts.
Photo of Dorian Perry

Dorian Perry, Employee

  • 2,300 Points 2k badge 2x thumb
Like Matthew suggested, if the switch can't communicate with the TACACS server, it may default to local credentials.
Have you tried disabling the TACACS server and attempting to login?
Photo of Mike D

Mike D, Alum

  • 3,852 Points 3k badge 2x thumb
Hello Michael,
Do you have another fabric - say a spare fabric with no config or a fabric robbed from another switch? Can you suffer some level of downtime to recover this?

One stability mechanism used in the distributed environment is a forced firmware and configuration copy from the chassis master fabric (master handles snmp, external auth etc and is identified by a green mgmt LED on the front panel) to any new module booting in the chassis.  With a 3rd fabric you could wipe out the configuration on each of the boogered up fabrics.  
This suggestion requires a maintenance window - less than an hour of outage I'd expect - given a little planning. Another caveat - access to a valid backup of your switch configuration - minus tacacs - will be needed for recovery. 

 
* Pull all fabrics and cards from the chassis.  
* Insert the new fabric (#3) into the chassis. This fabric will become chassis master.  
* Insert either of the fabrics with tacacs config - this will be erased.
* Pull out the spare fabric (#3) so the remaining fabric - now with erased config - becomes master.  At this stage you should be able to attach to management console. Log in with the default user/pass of admin/<enter> or the now copied fabric #3 creds.  
*Apply the backup configuration to begin recovery.  Copy and paste the configuration into console or configure snmp management access; use Netsite to transfer the config file. (or use ssh, ftp etc)
* Insert the remaining fabric so the new cleaned-up configuration will be inherited.  

If you have a better method of isolating either misconfigured fabric card in order to force a blank or known good config copy (a spare chassis and fabric for example), there's no point in strictly following these steps. Just make sure you control the master and use it to overwrite the bad configs.

If shutting off access to the tacacs server doesn't do the trick, the configuration overwrite steps or something similar should get the job done. 
Its clunky.  Its lengthy.  its drastic.  But its a plan-b in case plan-a shows up looking still less attractive.   

Regards
Mike
 
Photo of Mike D

Mike D, Alum

  • 3,852 Points 3k badge 2x thumb
Hello Michael,.

Another access method below.  Previous clunky approach is still valid but its a poor plan b.  Though a safe outage window is still required, the following steps make a far more attractive plan b.  Simpler is almost always better.

Start with access to a valid backup config to minimize down time needed for reconfiguration.

to clear nvram,

* Pull one of the fabrics out of the system.  

* Attach a console to the chassis and reset with the remaining fabric installed.  Power pull is probably the only option unless things have improved on site.

*Almost immediately upon system diags/post messaging, begin tapping keys on the keyboard to interrupt boot and gain access to the boot menu (system image loader prompt)  a bit like accessing the bios on a pc by interrupting boot process. The window to catch the interrupt is just a couple of seconds so it may take a couple of tries.

* Once at the system image loader prompt, a question mark will display a list of basic recovery commands, one of these will be clearnvram.  execute this.

*Following reset you should be able to login with default credentials.
admin/<enter> for superuser, public and private for snmp rw/ro.

*Plug in the second fabric module and the now cleaned-up chassis master should fix the second fabric.  If you can call no-configuration a fixed switch. 

*Reconfigure.
 
Mike
Photo of Mike D

Mike D, Alum

  • 3,852 Points 3k badge 2x thumb
Checking in Michael,  
Are we out of the woods yet?
Photo of Drew C.

Drew C., Community Manager

  • 38,276 Points 20k badge 2x thumb
Marking this as "Not a problem" since we don't know if it has been solved yet.  We can update status later if something changes.
Hey folks...

Sorry for not supply any reply for these last months, I have been very busy, but today I told to myself I had to give some explanation about what happened.

Here is what our team did:

We removed the main blade, and we have used the seventh jumper of this board to reset the running-config, after that I was able to recover my backup that was inside this slot.

But, thanks God, we had a backup file saved over there, and we did this procedure with this knowledge, otherwise we would have to reconfig the switch from the scratch all over again.

Thanks for all help I had here. You guys are doing a absolutely fenomenal job.
Photo of Drew C.

Drew C., Community Manager

  • 38,242 Points 20k badge 2x thumb
Thanks for coming back to update the thread, Michel.  I've marked it as "Solved."