[paramiko] How do I execute multiple commands in a singlechannel?

[Eric Noulard]

2009/3/31 James <rocketmonkeys at gmail.com>:
> On Mon, Mar 30, 2009 at 6:44 PM, Crooks, Sam <Sam.Crooks at experian.com> wrote:
>> Couldn't you also combine stdout and stderr using
>> http://www.lag.net/paramiko/docs/paramiko.Channel-class.html#set_combine_stderr
>
> Yes, that would fix the problem of checking both stderr and stdout.
> It wouldn't help with commands that have no output, however.  One
> example is that I did a 'cd directory_name' and there was no output.
> That means I have to know ahead of time if each command is going to
> have output, and if not, then don't do a blocking read().

If you

chan = SSHClient.invoke_shell(...)
and then

chan.send("cd dirname\n")

you may check that you receive "\n" on stdout (chan.recv(...))
and nothing else.

You should be able to check stderr (chan.recv_err(...)), this way you
know that your "cd"
command did succeed or not but for this you need to request an shell
WITHOUT a pty.

This is done by requesting the shell on the channel
(Channel.invoke_shell) without requesting a pty before that but I did
not tried that.

>>
>> This scheme is not working with "output-less" command.
>
> As Eric mentions above, you could run a command and then block with a
> timeout, and if you timeout then you know something went wrong.
> Except this has the same flaw; if the command doesn't have valid
> output,

Could you gives us an example of a command without output?
(I mean even "cd somedir" has one when invoke in an interactive
 session with pty since you'll get line return)

>  then you  never know if it ran or not.

If your command has NO output to stdout it must have done something useful
like writing a file on the remote side or any other side-effect.
The side-effect should be "checkable" by another command which
should have output.

Somehow you need to have a "set of" checkable command.
say that cmd1 has no output but cmd2 has and is able to check that previous
cmd1 did succeed then you may do:

chan = SSHClient.invoke_shell(...)
chan.send("cmd1\n")
chan.send("cmd2\n")

now you can check output of cmd2, the check should be done with timeout.

Note that even if you send both command without knowing "when"
cmd1 has ended it's not that bad because cmd2 won't start unless cmd1 has
ended when using an interactive shell.


> Eric, I see yours as a
> viable solution, but a messy one in my book.  It doesn't work for
> output-less commands, and it could potentially be slower than it needs
> to be since you're waiting for a timeout vs. knowing immediately if
> your command has succeeded or not.

As explained earlier, timeout is only useful for "dangling" command,
most of the time you don't wait because you'll have your "chekcing command"
which (immediatly) puts something on stdout.

"Immediately" knowing without output may be possible with
"exec_command" but again
you'll need timeout because your remotely executed command may be
blocked for unexpectedly long time.

I'm pretty sure timeout handling is unavoidable, but this is different
from knowing if the command succeeded or failed.

> I'm on a quest for the perfect
> solution here, and I know I'm going to have to compromise some way or
> another.  But it's good to discuss this all and get all the potential
> solutions out there, since some will work for some people, and may be
> enough.  I may be able to use one of the solutions here, I haven't
> tried them all yet.

I think at this point we need "concrete" example of runnable command
in order to testdrive potential solutions.

> I have to ask; right now we have a limitation that exec_command()
> closes a channel after it's done.  Is this a limitation of the SSH
> protocol, or of paramiko?  If we could change this limitation in
> paramiko so we could run exec_command() multiple times on a channel,
> it would fix a lot (if not all) of these problems.

I agree.
Seems like an SSH protocol thing
(see http://www.rfc-editor.org/rfc/rfc4254.txt §6.5)

> If it's an SSH
> protocol thing, then I guess we should be asking what the correct way
> to do this is via SSH, and then use that.  It sure doesn't seem simple though.

again: http://www.rfc-editor.org/rfc/rfc4254.txt §6

"A session is a remote execution of a program.  The program may be a
   shell, an application, a system command, or some built-in subsystem.
   It may or may not have a tty, and may or may not involve X11
   forwarding.  Multiple sessions can be active simultaneously."

which looks like each "session" has only "one" executed command.
the "shell" session is particular. The command you execute is a shell
thus it does not terminate unless you close it, which makes it
possible to "send" several command through it.

> James B: do you have a clear idea of how to add expect-like behavior
> to paramiko?  Do you need help doing this?  I'm just curious, I don't
> know if I'd have to time the spend on it but it sure seems like an
> interesting problem.

I'm pretty interested in this too.
Not that regarding the way to handle "expect-like" thing would be to
"port" pexpect to use paramiko.

Whatever the way you want to, expect-like feature needs some
"event" handling things on the channel WITH timeout feature.

Since SSH is a windowed protocol, see paramiko doc:

"Because SSH2 has a windowing kind of flow control, if you stop
reading data from a Channel and its buffer fills up, the server will
be unable to send you any more data until you read some of it. (This
won't affect other channels on the same transport -- all channels on a
single transport are flow-controlled independently.) Similarly, if the
server isn't reading data you send, calls to send may block, unless
you set a timeout. This is exactly like a normal network socket, so it
shouldn't be too surprising."

You'll need to handle a windows size in which you can "expect something".

Basically:

0) register expect/do actions
1) open a channel with appropriate window size
2) enter the loop in which you
      2.a) try to read more input and
      2.b) check if the current input buffer
              match an "expect rule"
              2.b.i) YES flush input buffer up to your matched expect rule
                        and do the corresponding action
              2.b.II) continue
      2.c) check for timeout and go to 2.a if no timeout occured


You may want to do that on multiple channels.

-- 
Erk