[paramiko] Intermittent problem (no attachments)
Todd Whiteman
toddw at activestate.com
Mon Aug 27 13:07:48 PDT 2007
Hi Jd,
First off, does this problem also exist when using the latest paramiko
release (1.7.1):
http://www.lag.net/paramiko/
Todd.
jd wrote:
> Hi
> I had sent this email, but is waiting for approval
> as the attached sources seems to cross the limit of
> the distribution list.
> Can the owner kindly approve the message with files.
> does anyone have tips/pointers to resolve this ?
> Thanks
> /Jd
>
> --- jd <jdsw2002 at yahoo.com> wrote:
>
>> Hi Everyone.
>> I have written a ssh tunnel transport for
>> XML-RPC.
>> It seems to work pretty good, but has intermittent
>> failures.
>>
>> It seems that the client/parser missed the first
>> chunk of data from the server.
>>
>> This does not happen when I use the normal xml-rpc
>> over http. so I am assuming that the server is ok.
>>
>> Attached is test program that reproduces the
>> problem.
>> Environment is FC7, python 2.5,
>> python-paramiko-1.6.4-1.fc7, Xen 3.1
>>
>> Can someone spot something wrong which might be
>> causing this ? Or some paramiko tweak that might be
>> useful in this scenario.
>>
>> Any other ideas on how to narrow down the problem
>> are
>> welcome.
>>
>> Thanks
>> /Jd
>>
>> === Attached description of files ===
>> phelper.py : Just some utility wrapper over paramiko
>> lib.
>> test_proxy.py : simple driver program that
>> reproduces
>> the problem.
>> test_output : output : 2 out of N tries succeed.
>> XenServreProxy.py : Proxy that implements the
>> paramiko
>> transport for XML-RPC.
>>
>>
>>
>>
>>
>>
>>
> ____________________________________________________________________________________
>> Sick sense of humor? Visit Yahoo! TV's
>> Comedy with an Edge to see what's on, when.
>> http://tv.yahoo.com/collections/222> #!/usr/bin/env
> python
>> #
>> # XenMan - Copyright (c) 2007 Jd & Hap Hazard
>> # ======
>> #
>> # XenMan is a Xen management tool with a GTK based
>> graphical interface
>> # that allows for performing the standard set of
>> domain operations
>> # (start, stop, pause, kill, shutdown, reboot,
>> snapshot, etc...). It
>> # also attempts to simplify certain aspects such as
>> the creation of
>> # domains, as well as making the consoles available
>> directly within the
>> # tool's user interface.
>> #
>> #
>> # This software is subject to the GNU Lesser General
>> Public License (LGPL)
>> # and for details, please consult it at:
>> #
>> # http://www.fsf.org/licensing/licenses/lgpl.txt
>> #
>> # author : Jd <jd_jedi at users.sourceforge.net>
>> #
>>
>> import paramiko
>> from paramiko import SSHException
>> import os, sys
>> import getpass
>> import socket
>>
>> """Paramiko helper class. Provides common functions
>> as
>> -- validating host keys,
>> -- initializing a new transport,
>> -- agent based and password based authentication
>> etc.
>> """
>> class HostValidationException(Exception):
>> def __init__(self, errno, description):
>> self.errno = errno
>> self.description = description
>>
>> def __repr__(self):
>> return "[Error %s], %s" % (self.errno,
>> self.description)
>>
>> def __str__(self):
>> return self.__repr__()
>>
>> class AuthenticationException(Exception):
>> def __init__(self, errno, description):
>> self.errno = errno
>> self.description = description
>>
>> def __repr__(self):
>> return "[Error %s], %s" % (self.errno,
>> self.description)
>>
>> def __str__(self):
>> return self.__repr__()
>>
>>
>> class CommunicationException(Exception):
>> def __init__(self, errno, description):
>> self.errno = errno
>> self.description = description
>>
>> def __repr__(self):
>> return "[Error %s], %s" % (self.errno,
>> self.description)
>>
>> def __str__(self):
>> return self.__repr__()
>>
>>
>>
>> class PHelper:
>>
>> host_keys = {}
>>
>> # credential helper
>> credentials_helper = None
>>
>> ## The credendital helper needs to
>> get_credentials(hostname) method
>> ## to return credentials
>> ## the object returned should:
>> ## get_username() and get_password() methods
>> ## This would be used when the transport can not
>> be initialized
>> ## using given methods
>>
>> @classmethod
>> def set_credentials_helper(cls, cred_helper):
>> """ Set the helper class"""
>> cls.credentials_helper = cred_helper
>>
>>
>> @classmethod
>> def load_keys(cls):
>> # TODO : May be we need to load
>> /etc/ssh/known_hosts and merge it here.
>> try:
>> path =
>> os.path.expanduser('~/.ssh/known_hosts')
>> cls.host_keys =
>> paramiko.util.load_host_keys(path)
>> except IOError:
>> try:
>> path =
>> os.path.expanduser('~/ssh/known_hosts')
>> cls.host_keys =
>> paramiko.util.load_host_keys(path)
>> except IOError:
>> pass
>>
>>
>> @classmethod
>> def init_log(cls,log_file_name):
>> paramiko.util.log_to_file(log_file_name)
>>
>>
>> @classmethod
>> def validate_host_key(cls, transport, hostname):
>> """
>> get the remote hosts key and validate
>> against known host keys
>> throws exception with errno, reason
>> errno - reason
>> 1 - Host not found
>> 2. - Host found but key not found
>> 3 - Authentication failed (wrong password?)
>> 4 - Host found, key found, but keys do not
>> match
>> (server changed/spoofed)
>> """
>> # check server's host key -- this is
>> important.
>> key = transport.get_remote_server_key()
>> if not PHelper.host_keys.has_key(hostname):
>> print "Warning : Host not found ! ",
>> hostname
>> #raise HostValidationException(1, "Host
>> not found")
>> elif not
>> PHelper.host_keys[hostname].has_key(key.get_name()):
>> print "Warning: Key not found ! ",
>> hostname
>> #raise HostValidationException(2, "Key
>> not found.")
>> elif
>> PHelper.host_keys[hostname][key.get_name()] != key:
>> raise HostValidationException(3, "Keys
>> mismatch for " + hostname)
>>
>> return True
>>
>>
>> ## TODO : only for testing purpose
>> @classmethod
>> def interactive_auth(cls, transport, username,
>> hostname):
>> default_auth = 'p'
>> auth = raw_input('Auth by (p)assword, (r)sa
>> key, or (d)ss key? [%s] ' % default_auth)
>> if len(auth) == 0:
>> auth = default_auth
>>
>> if auth == 'r':
>> default_path =
>> os.path.join(os.environ['HOME'], '.ssh', 'id_rsa')
>> path = raw_input('RSA key [%s]: ' %
>> default_path)
>> if len(path) == 0:
>> path = default_path
>> try:
>> key =
>> paramiko.RSAKey.from_private_key_file(path)
>> except
>> paramiko.PasswordRequiredException:
>> password = getpass.getpass('RSA key
>> password: ')
>> key =
>> paramiko.RSAKey.from_private_key_file(path,
>> password)
>> transport.auth_publickey(username, key)
>> elif auth == 'd':
>> default_path =
>> os.path.join(os.environ['HOME'], '.ssh', 'id_dsa')
>> path = raw_input('DSS key [%s]: ' %
>> default_path)
>> if len(path) == 0:
>> path = default_path
>> try:
>> key =
>> paramiko.DSSKey.from_private_key_file(path)
>> except
>> paramiko.PasswordRequiredException:
>>
> === message truncated ===> # This is a test program
> that reproduces the xml-rpc
>> over ssh_tunnel using
>> # paramiko.
>>
>> # The program works always when the protocol is
>> 'http' (python xml-rpc)
>> # but produces parsing errors when 'ssh_tunnel' is
>> used.
>>
>> # It seems that the client/parser is missing
>> first/some chunks
>> # and tries to parse from somewhere in the middle
>> producing the exception.
>>
>> # requires xen-utils / xen setup on the client
>> # requires xen setup on the server.
>>
>> from XenServerProxy import ServerProxy,
>> SSHTunnelTransport
>> from phelper import PHelper
>> import sys, traceback
>>
>> host = "192.168.12.103"
>> port = 8006
>> username = 'root'
>> password = "password"
>> protocol = "ssh_tunnel" # "http" or "ssh_tunnel"
>>
>> if True: # interactive
>> print "Host name : [" , host , "]"
>> res = sys.stdin.readline()
>> if (res.strip() != ''):
>> host = res.strip()
>>
>>
>>
>> print "Port to tunnel to : [" , port , "]"
>> res = sys.stdin.readline()
>> if (res.strip() != ''):
>> port = int(res.strip())
>>
>>
>> print "User name : [" , username , "]"
>> res = sys.stdin.readline()
>> if (res.strip() != ''):
>> username = res.strip()
>>
>>
>> print "Password : "
>> res = sys.stdin.readline()
>> if (res.strip() != ''):
>> password = res.strip()
>>
>>
>> print "Protocol : ssh_tunnel or http [" +
>> protocol + "] "
>> res = sys.stdin.readline()
>> if (res.strip() != ''):
>> protocol = res.strip()
>>
>>
>> ssh_transport = PHelper.init_ssh_transport(host,
>>
>> username=username,
>>
>> password=password)
>> if protocol == "ssh_tunnel":
>> tunnel_transport = SSHTunnelTransport(host,
>> port,
>>
>> ssh_transport = ssh_transport,
>> ssh_port =
>> 22)
>> else:
>> tunnel_transport = None
>> vmm = ServerProxy(protocol + '://' + username +
>> '@' +
>> host + ":" +
>> str(port) +
>> "/RPC2",
>> password=password,
>> transport=tunnel_transport
>> )
>>
>> while True:
>> try:
>> info = vmm.xend.domains()
>> print info
>> except Exception, ex:
>> print traceback.print_exc()
>>
>> print "Type q to quit, any other key to request
>> again"
>> res = sys.stdin.readline()
>> if (res.strip() == 'q'):
>> break
>>
>> ssh_transport.close()
>>
>>
>>
>>
>>> #!/usr/bin/env python
>> #
>> # XenMan - Copyright (c) 2007 Jd & Hap Hazard
>> # ======
>> #
>> # XenMan is a Xen management tool with a GTK based
>> graphical interface
>> # that allows for performing the standard set of
>> domain operations
>> # (start, stop, pause, kill, shutdown, reboot,
>> snapshot, etc...). It
>> # also attempts to simplify certain aspects such as
>> the creation of
>> # domains, as well as making the consoles available
>> directly within the
>> # tool's user interface.
>> #
>> #
>> # This software is subject to the GNU Lesser General
>> Public License (LGPL)
>> # and for details, please consult it at:
>> #
>> # http://www.fsf.org/licensing/licenses/lgpl.txt
>> #
>> # author : Jd <jd_jedi at users.sourceforge.net>
>> #
>>
>> from httplib import HTTPConnection, HTTP
>> from xmlrpclib import Transport
>>
>> from SimpleXMLRPCServer import SimpleXMLRPCServer,
>> SimpleXMLRPCRequestHandler
>> import xmlrpclib, socket, os, stat
>> import SocketServer
>>
>>
>> import xen.xend.XendClient
>> from xen.xend.XendLogging import log
>>
>>
>>
>> try:
>> from xen.util.xmlrpcclient import ServerProxy
>> base_pkg = xen.util.xmlrpcclient
>> except ImportError,e:
>> from xen.util.xmlrpclib2 import ServerProxy
>> base_pkg = xen.util.xmlrpclib2
>>
>> from getpass import getuser
>>
>> from phelper import PHelper
>> import xml.parsers.expat
>> # A new ServerProxy that also supports ssh tunneling
>> using paramiko.
>> # The ssh_tunnel URL comes in the
>> # form:
>> #
>> # ssh_tunnel://user@host:port/RPC2
>> #
>> # It assumes that the RPC handler is /RPC2.
>> # This is implemented in a fashion similar to other
>> transports in xmlrpcclient.
>> # Note : This one does not require corresponding
>> server side implementation
>> # as it uses the ssh tunneling mechanism. It
>> would be nice though to
>> # have another paramiko based server
>> implemeted and shipped by Xen
>> # to which we can directly connect.
>>
>>
>> DEFAULT_LOCAL_PORT_START = 2000
>> DEFAULT_LOCAL_PORT_END = 2000
>>
>> class SSHTunnelConnection(HTTPConnection):
>> def set_ssh_transport(self, ssh_transport):
>> self.ssh_transport = ssh_transport
>>
>> def set_local_port(self, localport):
>> self.localport = localport
>>
>> def connect(self):
>> # We already have a transport to the remote
>> machine.
>> # lets create a tunnel on the remote node to
>> forward traffic from
>> # localhost, localport ==> localhost, xen
>> port
>> # (NOTE : localhost is interpreted on the
>> remote node.)
>> self.sock =
>> PHelper.open_channel(self.ssh_transport,
>> "direct-tcpip",
>>
>> ("localhost",self.port),
>>
>> ("localhost",self.localport))
>>
>> class SSHTunnel(HTTP):
>>
>> _connection_class = SSHTunnelConnection
>>
>> def __init__(self, ssh_transport, host='',
>> port=None, strict=None,
>>
>> localport=DEFAULT_LOCAL_PORT_START):
>> self.localport = localport
>> self.ssh_transport = ssh_transport
>> HTTP.__init__(self, host, port, strict)
>>
>> # take this oppertunity to associate transport
>> with the connection.
>> def _setup(self, conn):
>> HTTP._setup(self, conn)
>> conn.set_ssh_transport(self.ssh_transport)
>> conn.set_local_port(self.localport)
>>
>>
>>
>> class SSHTunnelTransport(Transport):
>>
>> current_local_port = DEFAULT_LOCAL_PORT_START
>> _use_datetime = False
>>
>> def __init__(self,hostname,port,user=None,
>> password=None,
>> ssh_transport=None, ssh_port=22):
>> """ constructor, takes in host and port to
>> which the tunnel
>> is to be opened. Assumes ssh on 22 for
>> now.
>> An already authenticated paramiko
>> transport can be passed.
>> It is assumed that the transport is to
>> the same host as specified
>> by hostname.
>> """
>> self.hostname = hostname
>> self.ssh_port = ssh_port
>> self.username = user
>> self.port = port
>> self.transport_created=False
>> if ssh_transport == None:
>> self.transport =
>> PHelper.init_ssh_transport(self.hostname,
>>
>> self.ssh_port,
>>
>> username=self.username,
>>
>> password = password)
>> self.transport_created=True
>> else:
>> self.transport = ssh_transport
>>
>>
>> def request(self, host, handler, request_body,
>> verbose=0):
>> self.__handler = handler
>> return Transport.request(self, host,
>> '/RPC2', request_body, verbose)
>>
>> def make_connection(self, host):
>> """ return the connection paramiko connection
>> object """
>> SSHTunnelTransport.current_local_port += 1
>> if SSHTunnelTransport.current_local_port >=
>> DEFAULT_LOCAL_PORT_END:
>> SSHTunnelTransport.current_local_port =
>> DEFAULT_LOCAL_PORT_START
>>
>> return SSHTunnel(self.transport,host,
>>
>> localport=SSHTunnelTransport.current_local_port)
>>
>>
>> def cleanup(self):
>> """ clean up transport if we created it."""
>> if self.transport != None and
>> self.transport_created:
>> self.transport.close()
>>
>>
>>
>> ## copied _Method from xmlrpc and
>> class _Dispatcher:
>> # some magic to bind an XML-RPC method to an RPC
>> server.
>> # supports "nested" methods (e.g.
>> examples.getStateName)
>> def __init__(self, orig_object, send, name):
>> self.__send = send
>> self.__name = name
>> self.__orig_object = orig_object
>> def __getattr__(self, name):
>> return _Dispatcher(self.__orig_object,
>> self.__send, "%s.%s" %
>> (self.__name, name))
>> def __call__(self, *args):
>> try:
>> ret = self.__send(self.__name, args)
>>
> === message truncated ===
>
>
>
> ____________________________________________________________________________________
> Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7
>
>
> _______________________________________________
> paramiko mailing list
> paramiko at www.lag.net
> http://www.lag.net/cgi-bin/mailman/listinfo/paramiko
>
More information about the paramiko
mailing list