From jbardin at bu.edu Fri Dec 1 12:55:03 2006 From: jbardin at bu.edu (James Bardin) Date: Fri, 01 Dec 2006 15:55:03 -0500 Subject: [paramiko] Allowing multiple keys for pkey in SSHClient.connect Message-ID: <457096A7.1050804@bu.edu> Hi, I have been using sockets and the Transport up until now, but I'm now I'm trying to rewrite a program using the SSHClient abstraction provided. Is there a good method for testing multiple keys (especially from an agent) for authorization? running SSHClient.connect multiple time seems ugly, as the socket and transport are torn down and rebuilt each time. If this is the only way right now, does anyone see any problems with allowing pkey in SSHClient.connect to accept the tuple returned by Agent.get_keys() and test them all? thanks -jim -- /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * James Bardin - Systems Analyst / Administrator I Boston University Department of Electrical and Computer Engineering 8 Saint Mary's St, Room 305, Boston, MA 02215 Ph:617-358-2785 http://www.bu.edu/ece/it * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ From jbardin at bu.edu Fri Dec 1 13:35:42 2006 From: jbardin at bu.edu (James Bardin) Date: Fri, 01 Dec 2006 16:35:42 -0500 Subject: [paramiko] Allowing multiple keys for pkey in SSHClient.connect In-Reply-To: <457096A7.1050804@bu.edu> References: <457096A7.1050804@bu.edu> Message-ID: <4570A02E.4050705@bu.edu> Sorry everyone, I never thought to check if SSHClient queries the agent on it's own. Please ignore my previous post. -jim James Bardin wrote: > Hi, > > I have been using sockets and the Transport up until now, but I'm now > I'm trying to rewrite a program using the SSHClient abstraction provided. > > Is there a good method for testing multiple keys (especially from an > agent) for authorization? > running SSHClient.connect multiple time seems ugly, as the socket and > transport are torn down and rebuilt each time. > > If this is the only way right now, does anyone see any problems with > allowing pkey in SSHClient.connect to accept the tuple returned by > Agent.get_keys() and test them all? > > thanks > -jim > > -- /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * James Bardin - Systems Analyst / Administrator I Boston University Department of Electrical and Computer Engineering 8 Saint Mary's St, Room 305, Boston, MA 02215 Ph:617-358-2785 http://www.bu.edu/ece/it * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ From david at guerizec.net Sat Dec 2 09:53:01 2006 From: david at guerizec.net (David Guerizec) Date: Sat, 2 Dec 2006 18:53:01 +0100 Subject: [paramiko] typo in rev 417 (x11) Message-ID: <200612021853.01643.david@guerizec.net> Hi Robey, I've found a small typo in paramiko/transport.py that prevent X11 forwzrding from working. Following is the patch to apply on that file. Best regards, David === modified file 'paramiko/transport.py' --- paramiko/transport.py 2006-11-21 00:11:17 +0000 +++ paramiko/transport.py 2006-12-02 17:49:20 +0000 @@ -1345,7 +1345,7 @@ # by default, use the same mechanism as accept() self._x11_handler = self._default_x11_handler else: - self._x11_hanlder = handler + self._x11_handler = handler def _default_x11_handler(self, channel, (src_addr, src_port)): self._queue_incoming_channel(channel) From robey at lag.net Sat Dec 2 17:14:14 2006 From: robey at lag.net (Robey Pointer) Date: Sat, 2 Dec 2006 17:14:14 -0800 Subject: [paramiko] typo in rev 417 (x11) In-Reply-To: <200612021853.01643.david@guerizec.net> References: <200612021853.01643.david@guerizec.net> Message-ID: <318D5552-2093-48B5-984A-6A7AE0A6400D@lag.net> On 2 Dec 2006, at 9:53, David Guerizec wrote: > Hi Robey, > > I've found a small typo in paramiko/transport.py that prevent X11 > forwzrding > from working. Following is the patch to apply on that file. Good catch; thanks! :) robey From jbardin at bu.edu Sat Dec 2 17:45:27 2006 From: jbardin at bu.edu (james bardin) Date: Sat, 2 Dec 2006 20:45:27 -0500 Subject: [paramiko] path expanded twice in client.py Message-ID: Hi, just noticed that ~/.ssh is expanded and prefixed twice when looking for keys in client.py easy fix: --- paramiko/client.py 2006-11-24 12:11:36.000000000 -0500 +++ paramiko/client.py 2006-12-02 20:28:37.806657889 -0500 @@ -382,7 +382,6 @@ if os.path.isfile(dsa_key): keyfiles.append((DSSKey, dss_key)) for pkey_class, filename in keyfiles: - filename = os.path.expanduser('~/.ssh/' + filename) try: key = pkey_class.from_private_key_file(filename, password) self._log(DEBUG, 'Trying discovered key %s in %s' % (hexlify(key.get_fingerprint()), filename)) -- /* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * James Bardin - Systems Analyst / Administrator I Boston University Department of Electrical and Computer Engineering 8 Saint Mary's St, Room 305, Boston, MA 02215 Ph:617-358-2785 http://www.bu.edu/ece/it * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ From michael.gratton at rsp.com.au Sun Dec 10 16:53:16 2006 From: michael.gratton at rsp.com.au (Michael Gratton) Date: Mon, 11 Dec 2006 11:23:16 +1030 Subject: [paramiko] closing SFTP connections Message-ID: <457CABFC.1060508@rsp.com.au> Hi guys, We're using Paramiko for some SFTP transfers and are occasionally seeing connections not getting closed down correctly. This results in the python process seeming to hang. After doing an strace/lsof on such a process, it is continually select'ing an SSH connection. We are closing the connection by calling SFTPClient.close. Looking through the code base it seems that this calls Channel.close but not Transport.close. I'm going to try to work around the problem by calling Transport.close explicitly after closing the client, but is this something that should happen in SFTPClient.close automatically? Thanks, /MIke -- Michael Gratton -- Code Wrangler Rising Sun Pictures -- http://www.rsp.com.au/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://www.lag.net/pipermail/paramiko/attachments/20061211/2848be8d/attachment.pgp From mik3l3374 at gmail.com Sun Dec 10 20:23:19 2006 From: mik3l3374 at gmail.com (mike) Date: Mon, 11 Dec 2006 12:23:19 +0800 Subject: [paramiko] attribute error Message-ID: hi I am connecting to a SFTP server using only public key authentication here is my code: .... path = os.path.join("somepath", '.ssh', 'id_rsa_a') try: t = paramiko.Transport((svr, PORT)) t.connect(hostkey=None, username='', password=None,pkey=path) sftp = paramiko.SFTPClient.from_transport(t) try: sftp.put(fil) except Exception, v: print v except: some other codes........ and here is the error i get at t.connect(): DEB [20061211-12:07:50.238] thr=1 paramiko.transport: using kex diffie-hellman-group1-sha1; server key type ssh-dss; c ipher: local 3des-cbc, remote 3des-cbc; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none DEB [20061211-12:07:52.731] thr=1 paramiko.transport: Switch to new keys ... DEB [20061211-12:07:52.734] thr=2 paramiko.transport: Attempting public-key auth... DEB [20061211-12:07:52.948] thr=1 paramiko.transport: userauth is OK ERR [20061211-12:07:52.948] thr=1 paramiko.transport: Unknown exception: 'str' object has no attribute 'get_name' ERR [20061211-12:07:52.950] thr=1 paramiko.transport: Traceback (most recent call last): ERR [20061211-12:07:52.950] thr=1 paramiko.transport: File "/usr/python/lib/python2.4/site-packages/paramiko/transpo rt.py", line 1369, in run ERR [20061211-12:07:52.950] thr=1 paramiko.transport: File "/usr/python/lib/python2.4/site-packages/paramiko/auth_ha ndler.py", line 203, in _parse_service_accept ERR [20061211-12:07:52.951] thr=1 paramiko.transport: AttributeError: 'str' object has no attribute 'get_name' ERR [20061211-12:07:52.951] thr=1 paramiko.transport: May i know how to resolve this?? thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.lag.net/pipermail/paramiko/attachments/20061211/ee247647/attachment.htm From billzingler at gmail.com Tue Dec 12 12:25:36 2006 From: billzingler at gmail.com (Bill Zingler) Date: Tue, 12 Dec 2006 15:25:36 -0500 Subject: [paramiko] SFTP performance for large files Message-ID: All, I have been doing some benchmarks on Linux for transferring files over SSH. It appears that paramiko works relatively well for files less than 1 MB, but for larger files the transfer time as compared to scp for 100 MB and 1 GB becomes 20 times longer. Does anyone have any ideas as to why that is? Thanks, Bill -- Bill Zingler Zingler & Associates, Inc. (443) 822-4857 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.lag.net/pipermail/paramiko/attachments/20061212/97a3f4d7/attachment.htm From larstiq at larstiq.dyndns.org Wed Dec 13 05:08:35 2006 From: larstiq at larstiq.dyndns.org (Wouter van Heyst) Date: Wed, 13 Dec 2006 14:08:35 +0100 Subject: [paramiko] SFTP performance for large files In-Reply-To: References: Message-ID: <20061213130835.GA28151@larstiq.dyndns.org> On Tue, Dec 12, 2006 at 03:25:36PM -0500, Bill Zingler wrote: > All, > > I have been doing some benchmarks on Linux for transferring files over SSH. > It appears that paramiko works relatively well for files less than 1 MB, but > for larger files the transfer time as compared to scp for 100 MB and 1 GB > becomes 20 times longer. Does anyone have any ideas as to why that is? Not yet, but in order to get a better grip on it, does that happen for individual files only, or also if you transfer 100 1MB sized files? Wouter van Heyst -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: Digital signature Url : http://www.lag.net/pipermail/paramiko/attachments/20061213/e9baf905/attachment.pgp From mik3l3374 at gmail.com Wed Dec 13 07:41:27 2006 From: mik3l3374 at gmail.com (mike) Date: Wed, 13 Dec 2006 23:41:27 +0800 Subject: [paramiko] authentication failed Message-ID: hi i have this code from the demo. I wanted to connect using only public key authentication try: k = paramiko.RSAKey(filename='test_rsa') #test_rsa is priv key t = paramiko.Transport(("host", port)) t.connect(username="user",pkey=k) #using only public key authentication sftp = paramiko.SFTPClient.from_transport(t) # dirlist on remote host dirlist = sftp.listdir('.') print "Dirlist:", dirlist t.close() except Exception, e: print '*** Caught exception: %s: %s' % (e.__class__, e) traceback.print_exc() try: t.close() except: pass sys.exit(1) when i ran it, it give me this error: *** Caught exception: paramiko.AuthenticationException: Authentication failed. Traceback (most recent call last): File "./test.py", line 24, in ? t.connect(username="user",pkey=k) File "/usr/python/lib/python2.4/site-packages/paramiko/transport.py", line 854, in connect self.auth_publickey(username, pkey) File "/usr/python/lib/python2.4/site-packages/paramiko/transport.py", line 1081, in auth_publickey return self.auth_handler.wait_for_response(my_event) File "/usr/python/lib/python2.4/site-packages/paramiko/auth_handler.py", line 174, in wait_for_response raise e AuthenticationException: Authentication failed. the error log shows: remote aes128-cbc; mac: local hmac-sha1, remote hmac-sha1; compression: local none, remote none DEB [20061213-23:39:31.225] thr=1 paramiko.transport: Switch to new keys ... DEB [20061213-23:39:31.230] thr=2 paramiko.transport: Attempting public-key auth... DEB [20061213-23:39:31.318] thr=1 paramiko.transport: userauth is OK INF [20061213-23:39:31.699] thr=1 paramiko.transport: Authentication (publickey) failed. DEB [20061213-23:39:31.810] thr=1 paramiko.transport: EOF in transport thread What am i doing wrong? thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.lag.net/pipermail/paramiko/attachments/20061213/63c935d0/attachment.htm From billzingler at gmail.com Thu Dec 14 14:07:50 2006 From: billzingler at gmail.com (Bill Zingler) Date: Thu, 14 Dec 2006 17:07:50 -0500 Subject: [paramiko] SFTP performance for large files In-Reply-To: <20061213130835.GA28151@larstiq.dyndns.org> References: <20061213130835.GA28151@larstiq.dyndns.org> Message-ID: Hello Wouter, I have attached my test scripts which I'm using for testing secure (scp, sftp) and non-secure (ftp) file transfer clients on both Linux and Windows to a Linux server. All of my testing is being run over a dedicated Gigabit Ethernet between the client and server. I find that paramiko will run slower by a factor of 2 or 3 as compared to scp on Linux and pscp on Windows for files that are less then 1 MB. This is quite reasonable for pure python implementation compared to a C or C++ implementation that scp and pscp are. But as I tested file sizes in the 100 MB and 1 GB range, I find that paramiko will run slower by a factor of 20. I'm sure that there is some room for optimization. The testing is controlled by the .cfg file, so you can tailor it to change file sizes and repetitions. The script requires the ConfigObj module which you can get from the Cheese Shop. Other than that, the remaining dependencies should be standard ones. I would really like to use to use paramiko in my application and would if only small files were needed to be transferred. Thanks for the help, Bill On 12/13/06, Wouter van Heyst wrote: > > On Tue, Dec 12, 2006 at 03:25:36PM -0500, Bill Zingler wrote: > > All, > > > > I have been doing some benchmarks on Linux for transferring files over > SSH. > > It appears that paramiko works relatively well for files less than 1 MB, > but > > for larger files the transfer time as compared to scp for 100 MB and 1 > GB > > becomes 20 times longer. Does anyone have any ideas as to why that is? > > Not yet, but in order to get a better grip on it, does that happen for > individual files only, or also if you transfer 100 1MB sized files? > > Wouter van Heyst > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > > iQCVAwUBRX/7SenTGucjhaP+AQLQawP+KKcdfJLkaInQ7QrzCpnepKD0y/rO8Qjv > frvIBDLhYtyakeOQbRLWyrGoqxWMz4lI742O7xB/sQ5pRB5s5L1Icbhk8Hzn272s > mQT3+J7NzPvkJss+o+nDyyM7ZIFfeH/MVbiUNfFyDRjMlJ/IDj+Fz/h+FE5jpyZ9 > TreImQ+pRJA= > =Vx+I > -----END PGP SIGNATURE----- > > > _______________________________________________ > paramiko mailing list > paramiko at lag.net > http://www.lag.net/mailman/listinfo/paramiko > > -- Bill Zingler Zingler & Associates, Inc. (443) 822-4857 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.lag.net/pipermail/paramiko/attachments/20061214/25c3737a/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: filetransferbenchmark.py Type: text/x-python Size: 10476 bytes Desc: not available Url : http://www.lag.net/pipermail/paramiko/attachments/20061214/25c3737a/attachment.py -------------- next part -------------- A non-text attachment was scrubbed... Name: filetransferbenchmark.cfg Type: application/octet-stream Size: 1195 bytes Desc: not available Url : http://www.lag.net/pipermail/paramiko/attachments/20061214/25c3737a/attachment.obj From robey at lag.net Fri Dec 15 11:31:18 2006 From: robey at lag.net (Robey Pointer) Date: Fri, 15 Dec 2006 11:31:18 -0800 Subject: [paramiko] closing SFTP connections In-Reply-To: <457CABFC.1060508@rsp.com.au> References: <457CABFC.1060508@rsp.com.au> Message-ID: On 10 Dec 2006, at 16:53, Michael Gratton wrote: > Hi guys, > > We're using Paramiko for some SFTP transfers and are occasionally > seeing > connections not getting closed down correctly. This results in the > python process seeming to hang. After doing an strace/lsof on such a > process, it is continually select'ing an SSH connection. > > We are closing the connection by calling SFTPClient.close. Looking > through the code base it seems that this calls Channel.close but not > Transport.close. > > I'm going to try to work around the problem by calling Transport.close > explicitly after closing the client, but is this something that should > happen in SFTPClient.close automatically? An SSH transport can host more than one open channel at a time. For example, you could have two open SFTP sessions and a shell session all over the same transport. So having Channel or SFTPClient close the Transport might prematurely close other sessions. (I think we used to have a __del__ method on Transport, that would automatically close itself if it was garbage-collected, but python interacts badly with __del__ in many cases and I think we decided it was safer to not have it.) Since all Channel and SFTPClient objects belong to a single Transport, if you close the Transport (or SSHClient), it should automatically close all of its channels. That will let you still have only one "close()" call. Hope that helps1 robey From robey at lag.net Fri Dec 15 12:38:46 2006 From: robey at lag.net (Robey Pointer) Date: Fri, 15 Dec 2006 12:38:46 -0800 Subject: [paramiko] attribute error In-Reply-To: References: Message-ID: <7528A401-0660-4DA1-94C6-39D1483DE965@lag.net> On 10 Dec 2006, at 20:23, mike wrote: > hi > > I am connecting to a SFTP server using only public key authentication > > here is my code: > .... > path = os.path.join("somepath", '.ssh', 'id_rsa_a') > try: > t = paramiko.Transport((svr, PORT)) > t.connect(hostkey=None, username='', password=None,pkey=path) This is your problem ^ You need to create a key object from the ssh key file: key = paramiko.RSAKey.from_private_key_file(path) and pass that key object as the 'pkey' parameter to Transport.connect(). robey From robey at lag.net Fri Dec 15 14:23:15 2006 From: robey at lag.net (Robey Pointer) Date: Fri, 15 Dec 2006 14:23:15 -0800 Subject: [paramiko] closing SFTP connections In-Reply-To: References: <457CABFC.1060508@rsp.com.au> Message-ID: <2CDF6002-9B16-456E-BB53-FC3C2149F524@lag.net> On 15 Dec 2006, at 11:31, Robey Pointer wrote: > > On 10 Dec 2006, at 16:53, Michael Gratton wrote: > >> Hi guys, >> >> We're using Paramiko for some SFTP transfers and are occasionally >> seeing >> connections not getting closed down correctly. This results in the >> python process seeming to hang. After doing an strace/lsof on such a >> process, it is continually select'ing an SSH connection. >> >> We are closing the connection by calling SFTPClient.close. Looking >> through the code base it seems that this calls Channel.close but not >> Transport.close. >> >> I'm going to try to work around the problem by calling >> Transport.close >> explicitly after closing the client, but is this something that >> should >> happen in SFTPClient.close automatically? > > An SSH transport can host more than one open channel at a time. For > example, you could have two open SFTP sessions and a shell session > all over the same transport. So having Channel or SFTPClient close > the Transport might prematurely close other sessions. > > (I think we used to have a __del__ method on Transport, that would > automatically close itself if it was garbage-collected, but python > interacts badly with __del__ in many cases and I think we decided it > was safer to not have it.) > > Since all Channel and SFTPClient objects belong to a single > Transport, if you close the Transport (or SSHClient), it should > automatically close all of its channels. That will let you still > have only one "close()" call. Responding to myself here: It occurred to me later that SSHClient could attach the transport to a ResourceManager, so that when an SSHClient object is collected, the transport is closed. This appears to work. (It won't work for Transport objects themselves because they launch a thread.) I checked it in as revision 421, so if you're interested, you can try that out. It should mean that if you use SSHClient, when the SSHClient object is garbage collected, the underlying SSH transport is cleanly shutdown. robey From robey at lag.net Sat Dec 16 17:17:44 2006 From: robey at lag.net (Robey Pointer) Date: Sat, 16 Dec 2006 17:17:44 -0800 Subject: [paramiko] path expanded twice in client.py In-Reply-To: References: Message-ID: <115D5D44-9320-4042-989B-F311AEABBC76@lag.net> On 2 Dec 2006, at 17:45, james bardin wrote: > Hi, > > just noticed that ~/.ssh is expanded and prefixed twice when looking > for keys in client.py > easy fix: > > --- paramiko/client.py 2006-11-24 12:11:36.000000000 -0500 > +++ paramiko/client.py 2006-12-02 20:28:37.806657889 -0500 > @@ -382,7 +382,6 @@ > if os.path.isfile(dsa_key): > keyfiles.append((DSSKey, dss_key)) > for pkey_class, filename in keyfiles: > - filename = os.path.expanduser('~/.ssh/' + filename) > try: > key = pkey_class.from_private_key_file(filename, > password) > self._log(DEBUG, 'Trying discovered key %s in %s' % > (hexlify(key.get_fingerprint()), filename)) I think you're right -- thanks! robey From robey at lag.net Sat Dec 16 19:13:09 2006 From: robey at lag.net (Robey Pointer) Date: Sat, 16 Dec 2006 19:13:09 -0800 Subject: [paramiko] Paramiko and win-ssh-agent In-Reply-To: <456B4BAE.1070301@gmail.com> References: <456B4BAE.1070301@gmail.com> Message-ID: <0702F9C2-0352-4FE2-A7A7-89335DAE2499@lag.net> On 27 Nov 2006, at 12:33, Patrick M?zard wrote: > Hello, > I had to use paramiko in combination with win-ssh-agent ( www.ganaware.jp/archives/2006/04/winsshaskpass_1.html>), which is > thin wrapper for cygwin ssh-agent. Patch is attached. > > Basically, I wrote something compatible with cygwin unix domain > sockets and patch it in the existing paramiko.Agent implementation. > It works for me so... I think the implementation has the following > drawbacks: > - completely tied to cygwin unix socket implementation. I think > this is OK, the handshake protocol have been stable for at least a > year. > - cygwin module reads cygwin /etc/passwd at load time to get user > id and group id. I assumed people have the permissions to read > this, probably OK, at least in a cygwin context. > - cygwin root directory is read from PARAMIKO_CYGPATH. It could > probably be done in smarter ways, but it works and is simple. Good > enough for now, will not be hard to improve if necessary. There's a lot of cygwin magic in this patch. Can't cygwin just implement the local-domain socket API and hide all of these details in there? The stuff about parsing the socket file just looks really fragile. robey From robey at lag.net Sat Dec 16 20:05:45 2006 From: robey at lag.net (Robey Pointer) Date: Sat, 16 Dec 2006 20:05:45 -0800 Subject: [paramiko] authentication failed In-Reply-To: References: Message-ID: On 13 Dec 2006, at 7:41, mike wrote: > hi > i have this code from the demo. I wanted to connect using only > public key authentication > try: > k = paramiko.RSAKey(filename='test_rsa') #test_rsa is priv key > t = paramiko.Transport(("host", port)) > t.connect(username="user",pkey=k) #using only public key > authentication > sftp = paramiko.SFTPClient.from_transport(t) > > # dirlist on remote host > dirlist = sftp.listdir('.') > print "Dirlist:", dirlist > t.close() > > except Exception, e: > print '*** Caught exception: %s: %s' % (e.__class__, e) > traceback.print_exc() > try: > t.close() > except: > pass > sys.exit(1) > > > when i ran it, it give me this error: > > *** Caught exception: paramiko.AuthenticationException: > Authentication failed. > Traceback (most recent call last): > File "./test.py", line 24, in ? > t.connect(username="user",pkey=k) > File "/usr/python/lib/python2.4/site-packages/paramiko/ > transport.py", line 854, in connect > self.auth_publickey(username, pkey) > File "/usr/python/lib/python2.4/site-packages/paramiko/ > transport.py", line 1081, in auth_publickey > return self.auth_handler.wait_for_response(my_event) > File "/usr/python/lib/python2.4/site-packages/paramiko/ > auth_handler.py", line 174, in wait_for_response > raise e > AuthenticationException: Authentication failed. > > > the error log shows: > > remote aes128-cbc; mac: local hmac-sha1, remote hmac-sha1; > compression: local none, remote none > DEB [20061213-23:39:31.225] thr=1 paramiko.transport: Switch to > new keys ... > DEB [20061213-23:39:31.230] thr=2 paramiko.transport: Attempting > public-key auth... > DEB [20061213-23:39:31.318] thr=1 paramiko.transport: userauth is OK > INF [20061213-23:39:31.699] thr=1 paramiko.transport: > Authentication (publickey) failed. > DEB [20061213-23:39:31.810] thr=1 paramiko.transport: EOF in > transport thread > > What am i doing wrong? > thanks Maybe the error message could be clearer. It means your login was rejected by the server. robey From pmezard at gmail.com Sun Dec 17 02:06:34 2006 From: pmezard at gmail.com (=?ISO-8859-1?Q?Patrick_M=E9zard?=) Date: Sun, 17 Dec 2006 11:06:34 +0100 Subject: [paramiko] Paramiko and win-ssh-agent In-Reply-To: <0702F9C2-0352-4FE2-A7A7-89335DAE2499@lag.net> References: <456B4BAE.1070301@gmail.com> <0702F9C2-0352-4FE2-A7A7-89335DAE2499@lag.net> Message-ID: <458516AA.4070104@gmail.com> Robey Pointer wrote: > > On 27 Nov 2006, at 12:33, Patrick M?zard wrote: >> Basically, I wrote something compatible with cygwin unix domain >> sockets and patch it in the existing paramiko.Agent implementation. It >> works for me so... I think the implementation has the following >> drawbacks: >> - completely tied to cygwin unix socket implementation. I think this >> is OK, the handshake protocol have been stable for at least a year. >> - cygwin module reads cygwin /etc/passwd at load time to get user id >> and group id. I assumed people have the permissions to read this, >> probably OK, at least in a cygwin context. >> - cygwin root directory is read from PARAMIKO_CYGPATH. It could >> probably be done in smarter ways, but it works and is simple. Good >> enough for now, will not be hard to improve if necessary. > > There's a lot of cygwin magic in this patch. Can't cygwin just > implement the local-domain socket API and hide all of these details in > there? The stuff about parsing the socket file just looks really fragile. All the black magic is hidden in cygwin.py, the only reason to patch agent.py is I cannot provide non-connected sockets. I do not really get what you mean by "Can't cygwin just implement the local-domain socket API and hide all of these details in there?". Which cygwin are you talking about: the python module or the project ? Could you elaborate on the code changes you have in mind ? I agree that all the hand-shaking stuff relies on Cygwin implementation details but I do not see how it can be done otherwise, but by asking cygwin developers to keep it stable... I agree it might be too fragile for you to integrate in Paramiko as such. At least people could find the patch on the mailing list if they ever need it. -- Patrick M?zard From robey at lag.net Sun Dec 17 13:57:40 2006 From: robey at lag.net (Robey Pointer) Date: Sun, 17 Dec 2006 13:57:40 -0800 Subject: [paramiko] SFTP performance for large files In-Reply-To: References: <20061213130835.GA28151@larstiq.dyndns.org> Message-ID: On 14 Dec 2006, at 14:07, Bill Zingler wrote: > Hello Wouter, > > I have attached my test scripts which I'm using for testing secure > (scp, sftp) and non-secure (ftp) file transfer clients on both > Linux and Windows to a Linux server. All of my testing is being > run over a dedicated Gigabit Ethernet between the client and server. > > I find that paramiko will run slower by a factor of 2 or 3 as > compared to scp on Linux and pscp on Windows for files that are > less then 1 MB. This is quite reasonable for pure python > implementation compared to a C or C++ implementation that scp and > pscp are. But as I tested file sizes in the 100 MB and 1 GB range, > I find that paramiko will run slower by a factor of 20. I'm sure > that there is some room for optimization. > > The testing is controlled by the .cfg file, so you can tailor it to > change file sizes and repetitions. The script requires the > ConfigObj module which you can get from the Cheese Shop. Other > than that, the remaining dependencies should be standard ones. > > I would really like to use to use paramiko in my application and > would if only small files were needed to be transferred. I tried this on my local (home) network, from a macbook pro (laptop) to a Linux box, but got different results: the transfer rate slightly improves as the file size gets larger. Actually this is what I would expect, since the initial handshake is included in the timing, and for smaller files, that will be significiant. ----- paramiko took 10.501155 seconds to write a 1MB file 10 times with a transfer rate of 0.952276 MB and avg xfer time of 1.050115 sec paramiko took 76.212235 seconds to write a 10MB file 10 times with a transfer rate of 1.312125 MB and avg xfer time of 7.621223 sec paramiko took 152.376369 seconds to write a 20MB file 10 times with a transfer rate of 1.312539 MB and avg xfer time of 15.237637 sec paramiko took 368.105170 seconds to write a 50MB file 10 times with a transfer rate of 1.358307 MB and avg xfer time of 36.810517 sec paramiko took 764.182173 seconds to write a 100MB file 10 times with a transfer rate of 1.308588 MB and avg xfer time of 76.418217 sec paramiko took 1466.026837 seconds to write a 200MB file 10 times with a transfer rate of 1.364232 MB and avg xfer time of 146.602684 sec ----- One thing to note: I'm using a modified version of pyCrypto, from here: https://launchpad.net/people/robey/+branch/pycrypto/devel which fixes a performance problem in the crypto layer. (I should probably do a new release soon.) I don't think that would affect large files disproportionately, though, it just makes all transfers slower -- hence the 2-3x slowdown you're seeing. What kind of network were you trying over? If it's a high-latency link, paramiko might be filling up memory with pipelined data, and maybe we should limit that. Currently it will dump as much data as it can. Attached is the test script I used, in case there's something I left out. It's basically a truncated version of the one you posted, but only using paramiko. If you have lsprof installed, and you change 'xfer' to 'xfer_profiled' in the script, it will profile the put() call and generate call traces, which might add useful info. robey -------------- next part -------------- A non-text attachment was scrubbed... Name: perf.zip Type: application/zip Size: 8346 bytes Desc: not available Url : http://www.lag.net/pipermail/paramiko/attachments/20061217/7ccdcb3f/attachment.zip From robey at lag.net Sun Dec 17 14:01:39 2006 From: robey at lag.net (Robey Pointer) Date: Sun, 17 Dec 2006 14:01:39 -0800 Subject: [paramiko] Paramiko and win-ssh-agent In-Reply-To: <458516AA.4070104@gmail.com> References: <456B4BAE.1070301@gmail.com> <0702F9C2-0352-4FE2-A7A7-89335DAE2499@lag.net> <458516AA.4070104@gmail.com> Message-ID: <8BE51754-93A4-4283-878C-3068AB56695C@lag.net> On 17 Dec 2006, at 2:06, Patrick M?zard wrote: > Robey Pointer wrote: >> On 27 Nov 2006, at 12:33, Patrick M?zard wrote: >>> Basically, I wrote something compatible with cygwin unix domain >>> sockets and patch it in the existing paramiko.Agent >>> implementation. It works for me so... I think the implementation >>> has the following drawbacks: >>> - completely tied to cygwin unix socket implementation. I think >>> this is OK, the handshake protocol have been stable for at least >>> a year. >>> - cygwin module reads cygwin /etc/passwd at load time to get user >>> id and group id. I assumed people have the permissions to read >>> this, probably OK, at least in a cygwin context. >>> - cygwin root directory is read from PARAMIKO_CYGPATH. It could >>> probably be done in smarter ways, but it works and is simple. >>> Good enough for now, will not be hard to improve if necessary. >> There's a lot of cygwin magic in this patch. Can't cygwin just >> implement the local-domain socket API and hide all of these >> details in there? The stuff about parsing the socket file just >> looks really fragile. > > All the black magic is hidden in cygwin.py, the only reason to > patch agent.py is I cannot provide non-connected sockets. > > I do not really get what you mean by "Can't cygwin just implement > the local-domain socket API and hide all of these details in > there?". Which cygwin are you talking about: the python module or > the project ? Could you elaborate on the code changes you have in > mind ? I agree that all the hand-shaking stuff relies on Cygwin > implementation details but I do not see how it can be done > otherwise, but by asking cygwin developers to keep it stable... What I mean is: IMHO, the details of how to emulate a local-domain socket are best left inside the cygwin module -- either in their "libc" or in the python wrapper. In other words, I think a socket (AF_LOCAL, ...) call should be caught by the cygwin libraries and handled accordingly. Does that not work? I just imagine that paramiko is not the only python code out there that tries to use local-domain sockets... :) > I agree it might be too fragile for you to integrate in Paramiko as > such. At least people could find the patch on the mailing list if > they ever need it. Agreed, it's a useful patch in that respect. I'm just hoping it could be made much smaller and included in paramiko directly. robey From michael.gratton at rsp.com.au Sun Dec 17 15:03:22 2006 From: michael.gratton at rsp.com.au (Michael Gratton) Date: Mon, 18 Dec 2006 09:33:22 +1030 Subject: [paramiko] closing SFTP connections In-Reply-To: References: <457CABFC.1060508@rsp.com.au> Message-ID: <4585CCBA.4090408@rsp.com.au> Hi Robey, Robey Pointer wrote: > > An SSH transport can host more than one open channel at a time. For > example, you could have two open SFTP sessions and a shell session > all over the same transport. So having Channel or SFTPClient close > the Transport might prematurely close other sessions. You know, just after sending this mail, I actually managed work to that out myself. :) I think the source of confusion was that SSHClient.close /does/ close the transport and because SFTPClient is named similarly, I was expecting it to work in the same way. Maybe it could be named SFTPChannel or something instead... but I see it's already been renamed once. > Since all Channel and SFTPClient objects belong to a single > Transport, if you close the Transport (or SSHClient), it should > automatically close all of its channels. That will let you still > have only one "close()" call. Yep, but should the transport also automatically close once all channels are closed? We are seeing this happen most times, but for something like 1 in 20 connections, if all channels are closed (we're only opening one per transport) the transport isn't closed. Cheers, /Mike -- Michael Gratton -- Code Wrangler Rising Sun Pictures -- http://www.rsp.com.au/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://www.lag.net/pipermail/paramiko/attachments/20061218/1d83323c/attachment.pgp From robey at lag.net Wed Dec 27 14:01:48 2006 From: robey at lag.net (Robey Pointer) Date: Wed, 27 Dec 2006 17:01:48 -0500 Subject: [paramiko] Don't catch AssertionError In-Reply-To: <45920A05.9030005@arbash-meinel.com> References: <458AE9B5.6010100@arbash-meinel.com> <69572289-5CBB-4E98-9117-951F06C3692D@lag.net> <45920A05.9030005@arbash-meinel.com> Message-ID: [cross-posting to paramiko] On 27 Dec 2006, at 0:52, John Arbash Meinel wrote: > Robey Pointer wrote: >> >> On 21 Dec 2006, at 15:08, John Arbash Meinel wrote: >> >>> I just wanted to remind everyone that if you find yourself writing a >>> test that catches AssertionError, it is usually a sign that >>> something is >>> wrong with the code. >>> >>> I know of one place where we *have* to. But it is because >>> paramiko has >>> an assert, rather than something we are doing. >> >> I just grepped bzrlib for this and I think the assert was in >> transport/sftp.py, not paramiko, so we can easily avoid doing it. >> Here's a quick bundle that replaces the (commented-out) assert with a >> PathNotChild exception, and uncomments the corresponding unit tests. >> The tests all still pass. >> >> robey > > Well, you are correct that this can be changed as you say it is. > > But my comment was regarding around line 455 in sftp_file.py: > > [...] > > The 'assert length == len(data)' is making sure that the returned data > is the exact same length as what was requested. Which could be > violated > if you allowed requests >32KB, or if there was a readv for 100 bytes, > and there is only 80 bytes left in the file. > > paramiko shouldn't be using 'assert' to check an invariant in > something > that it is reading from the real world. It should be doing a real > check, > and raising a real error if it fails. Hmmm... Requests over 32K should be caught elsewhere, but it's true that EOF could cause a short response. Originally the assert was meant to verify that the readv tables were internally consistent, but I guess there are other checks on that now (plus unit tests, of course). I'll remove that assert. robey From cktan at greenplum.com Thu Dec 28 17:27:15 2006 From: cktan at greenplum.com (CK Tan) Date: Thu, 28 Dec 2006 17:27:15 -0800 Subject: [paramiko] client.py in version 1.6.4 Message-ID: <49D07A08-0706-449D-BDD6-D235FF75639F@greenplum.com> In reference to bug# 69330 ("client.py doesn't handle nonexistant key files https://bugs.launchpad.net/products/paramiko/+bug/69330), the bug fix introduced another bug: keyfiles = [] rsa_key = os.path.expanduser('~/.ssh/id_rsa') dsa_key = os.path.expanduser('~/.ssh/is_dsa') if os.path.isfile(rsa_key): keyfiles.append((RSAKey, rsa_key)) if os.path.isfile(dsa_key): keyfiles.append((DSSKey, dss_key)) for pkey_class, filename in keyfiles: ---> filename = os.path.expanduser(''~/.ssh/' + filename) try: key = pkey_class.from_private_key_file(filename, password) self._log(DEBUG, 'Trying discovered key %s in %s' % (hexlify(key.get_fingerprint()), filename)) self._transport.auth_publickey(username, key) return except SSHException, e: saved_exception = e except IOError, e: saved_exception = e The indicated line above may result in ~/.ssh/ being expanded more than once, resulting in the error below: % python Python 2.3.5 (#1, Aug 19 2006, 21:31:42) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import paramiko >>> paramiko.__version__ '1.6.4 (yanma)' >>> s = paramiko.SSHClient() >>> s.load_system_host_keys() >>> s.connect('localhost') Traceback (most recent call last): File "", line 1, in ? File "/Users/cktan/w/cdb2/mppMgmt/bin/lib/paramiko/client.py", line 277, in connect self._auth(username, password, pkey, key_filename) File "/Users/cktan/w/cdb2/mppMgmt/bin/lib/paramiko/client.py", line 405, in _auth raise saved_exception IOError: [Errno 2] No such file or directory: '/Users/cktan/.ssh// Users/cktan/.ssh/id_rsa' >>> Suggested fix: condition the clause by if filename.find('/.ssh/') >= 0: filename = os.path.expanduser ('~/.ssh/' + filename) From robey at lag.net Sat Dec 30 22:42:12 2006 From: robey at lag.net (Robey Pointer) Date: Sat, 30 Dec 2006 22:42:12 -0800 Subject: [paramiko] client.py in version 1.6.4 In-Reply-To: <49D07A08-0706-449D-BDD6-D235FF75639F@greenplum.com> References: <49D07A08-0706-449D-BDD6-D235FF75639F@greenplum.com> Message-ID: <6C482839-55A0-4C71-BC43-00C6D5B21154@lag.net> On 28 Dec 2006, at 17:27, CK Tan wrote: > In reference to bug# 69330 ("client.py doesn't handle nonexistant key > files https://bugs.launchpad.net/products/paramiko/+bug/69330), > the bug fix introduced another bug: > > keyfiles = [] > rsa_key = os.path.expanduser('~/.ssh/id_rsa') > dsa_key = os.path.expanduser('~/.ssh/is_dsa') > if os.path.isfile(rsa_key): > keyfiles.append((RSAKey, rsa_key)) > if os.path.isfile(dsa_key): > keyfiles.append((DSSKey, dss_key)) > for pkey_class, filename in keyfiles: > ---> filename = os.path.expanduser(''~/.ssh/' + filename) > try: > key = pkey_class.from_private_key_file(filename, > password) > self._log(DEBUG, 'Trying discovered key %s in %s' % > (hexlify(key.get_fingerprint()), filename)) > self._transport.auth_publickey(username, key) > return > except SSHException, e: > saved_exception = e > except IOError, e: > saved_exception = e > > The indicated line above may result in ~/.ssh/ being expanded more > than once, > resulting in the error below: > > % python > Python 2.3.5 (#1, Aug 19 2006, 21:31:42) > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> import paramiko >>>> paramiko.__version__ > '1.6.4 (yanma)' >>>> s = paramiko.SSHClient() >>>> s.load_system_host_keys() >>>> s.connect('localhost') > Traceback (most recent call last): > File "", line 1, in ? > File "/Users/cktan/w/cdb2/mppMgmt/bin/lib/paramiko/client.py", > line 277, in connect > self._auth(username, password, pkey, key_filename) > File "/Users/cktan/w/cdb2/mppMgmt/bin/lib/paramiko/client.py", > line 405, in _auth > raise saved_exception > IOError: [Errno 2] No such file or directory: '/Users/cktan/.ssh// > Users/cktan/.ssh/id_rsa' >>>> > > > Suggested fix: > > condition the clause by > if filename.find('/.ssh/') >= 0: filename = os.path.expanduser > ('~/.ssh/' + filename) I believe this is fixed in the trunk, by removing the extra expanduser call. robey