Receiving "* IDLE TIMEOUT *" when running commands over SSH via bastion/jump server

AlfredForbrich — Mon, 27 Nov 2023 13:20:07 GMT

Hi,

I need to automate execution of some of the "show ***" command, i.e. "show clock", on the switches in my network, SLX-OS 20.5.1b, but need to ssh to the switch via a bastion/jump server. I'm using python paramiko module to ssh and automate the running of the scripts. Unfortunately, while I appear to be connecting to the remote site and can login, my command, i.e. "show clock" is never executed and I end up with the following output ( I've redacted site info with <text> )

Contacting: 1<slx ip>
['show clock Last login: Mon Nov 27 05:14:53 2023 from <bastion IP>\r\n', 'Unsuccessful attempts: 0 since last login.\r\n', '\r\n', 'SECURITY WARNING: The default password for at least\r\n', 'one default account (root, admin and user) have not been changed.\r\n', '\r\n', 'Welcome to the Extreme SLX-OS Software\r\n', 'admin connected from <bastion IP> using ssh on N170SLX-BLB\r\n', '\x1b[?7h<SLX Name># \r\n', '*** IDLE TIMEOUT ***\r\n']

I have searched the internet and found this similar issue and tried to use asyncio and asyncssh but I still end up with a failure.

https://github.com/ronf/asyncssh/issues/567

DEBUG:asyncio:Using selector: EpollSelector
<bastion IP> 22 <bastion user> <bastion password> <slx IP> 22 <slx user> <slx password>
DEBUG:asyncssh:Reading config from "/home/cna/.ssh/config"
DEBUG:asyncssh:Reading config from "/home/cna/.ssh/config.d/...."
INFO:asyncssh:Opening SSH connection to <bastion IP>, port 22
INFO:asyncssh:[conn=0] Connected to SSH server at <bastion IP>, port 22
INFO:asyncssh:[conn=0] Local address: 10.233.70.115, port 39820
INFO:asyncssh:[conn=0] Peer address: <bastion IP>, port 22
DEBUG:asyncssh:[conn=0] Sending version SSH-2.0-AsyncSSH_2.14.1
DEBUG:asyncssh:[conn=0] Received version SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.7
DEBUG:asyncssh:[conn=0] Requesting key exchange
DEBUG:asyncssh:[conn=0] Received key exchange request
DEBUG:asyncssh:[conn=0] Beginning key exchange
DEBUG:asyncssh:[conn=0] Completed key exchange
INFO:asyncssh:[conn=0] Beginning auth for user extreme
DEBUG:asyncssh:[conn=0] Trying public key auth with rsa-sha2-256 key
DEBUG:asyncssh:[conn=0] Signing request with rsa-sha2-256 key
INFO:asyncssh:[conn=0] Auth for user extreme succeeded
DEBUG:asyncssh:Reading config from "/home/..."
DEBUG:asyncssh:Reading config from "/home/....."
INFO:asyncssh:[conn=0] Opening SSH connection to <slx IP>, port 22 via SSH tunnel
INFO:asyncssh:[conn=0] Opening direct TCP connection to <slx IP>, port 22
INFO:asyncssh:[conn=0] Client address: dynamic port
DEBUG:asyncssh:[conn=0, chan=0] Set write buffer limits: low-water=16384, high-water=65536
DEBUG:asyncssh:[conn=0] Received unknown global request: hostkeys-00@openssh.com
INFO:asyncssh:[conn=1] Connected to SSH server at <slx IP>, port 22
INFO:asyncssh:[conn=1] Local address: 10.233.70.115, port 39820
INFO:asyncssh:[conn=1] Peer address: dynamic port
DEBUG:asyncssh:[conn=1] Sending version SSH-2.0-AsyncSSH_2.14.1
DEBUG:asyncssh:[conn=1] Received version SSH-2.0-OpenSSH_7.5 PKIX[10.2]
DEBUG:asyncssh:[conn=1] Requesting key exchange
DEBUG:asyncssh:[conn=1] Received key exchange request
DEBUG:asyncssh:[conn=1] Beginning key exchange
DEBUG:asyncssh:[conn=1] Completed key exchange
INFO:asyncssh:[conn=1] Beginning auth for user admin
DEBUG:asyncssh:[conn=1] Received authentication banner
DEBUG:asyncssh:[conn=1] Trying public key auth with rsa-sha2-256 key
DEBUG:asyncssh:[conn=1] Trying password auth
INFO:asyncssh:[conn=1] Auth for user admin succeeded
DEBUG:asyncssh:[conn=1, chan=0] Set write buffer limits: low-water=16384, high-water=65536
INFO:asyncssh:[conn=1, chan=0] Requesting new SSH session
DEBUG:asyncssh:[conn=1] Received unknown global request: hostkeys-00@openssh.com
INFO:asyncssh:[conn=1, chan=0] Command: show clock
Result 1 stdout in 0:00:00.070467: Last login: Mon Nov 27 14:14:25 2023 from 192.168.246.11
Unsuccessful attempts: 0 since last login.

INFO:asyncssh:[conn=1, chan=0] Closing channel
INFO:asyncssh:[conn=1, chan=0] Received exit status 0
INFO:asyncssh:[conn=1, chan=0] Received channel close
INFO:asyncssh:[conn=1, chan=0] Channel closed
INFO:asyncssh:[conn=1] Closing connection
INFO:asyncssh:[conn=1] Sending disconnect: Disconnected by application (11)
INFO:asyncssh:[conn=0, chan=0] Aborting channel
INFO:asyncssh:[conn=1] Connection closed
INFO:asyncssh:[conn=0] Closing connection
INFO:asyncssh:[conn=0, chan=0] Closing channel
INFO:asyncssh:[conn=0] Sending disconnect: Disconnected by application (11)
INFO:asyncssh:[conn=0] Connection closed
INFO:asyncssh:[conn=0, chan=0] Closing channel due to connection close
INFO:asyncssh:[conn=0, chan=0] Channel closed

Note that I'm able to ssh to the bastion and through the bastion to the switch and run commands but this is not possible in the production environment.

Re: Receiving "* IDLE TIMEOUT *" when running commands over SSH via bastion/jump serve

AlfredForbrich — Mon, 27 Nov 2023 14:57:29 GMT

It appears that I can work around my issue using the paramiko invoke_shell() function

proxy_jump_command='ssh -W {HOST}:{PORT} <bastion ssh info>'.format(HOST=slxAddress,PORT='22')
proxy=paramiko.ProxyCommand(proxy_jump_command)

try:
ssh_client.connect(slxAddress, username='<slx user>', password='<slx user password>', sock=proxy )

except Exception as e:
print("exception: {}".format(e))

if ssh_client.get_transport() is not None:
ssh_client.get_transport().is_active()
if ( debug 😞 print("connected: {}".format(ssh_client.get_transport().is_active()))
remote_conn = ssh_client.invoke_shell()
remote_conn.send('\r')
output = remote_conn.recv(65535)
time.sleep(0.5)

for cmd in slxcli_command_list :
cmd = cmd + "\r"
try:
remote_conn.send(cmd)
time.sleep(5.0)
output = remote_conn.recv(65535)
except Exception as e:
print("Exception: {}".format(e))
print("output: {}".format(output))
ssh_client.close()

topic Receiving "*** IDLE TIMEOUT ***" when running commands over SSH via bastion/jump server in Data Center (SLX)

Receiving "*** IDLE TIMEOUT ***" when running commands over SSH via bastion/jump server

Re: Receiving "*** IDLE TIMEOUT ***" when running commands over SSH via bastion/jump serve

topic Receiving "* IDLE TIMEOUT *" when running commands over SSH via bastion/jump server in Data Center (SLX)

Receiving "* IDLE TIMEOUT *" when running commands over SSH via bastion/jump server

Re: Receiving "* IDLE TIMEOUT *" when running commands over SSH via bastion/jump serve