[python] Verify host key with pysftp

I am writing a program using pysftp, and it wants to verify the SSH host Key against C:\Users\JohnCalvin\.ssh\known_hosts.

Using PuTTY, the terminal program is saving it to the Registry [HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\SshHostKeys].

How do I reconcile the difference between pysftp and PuTTY?

My code is:

import pysftp as sftp

def push_file_to_server():
    s = sftp.Connection(host='138.99.99.129', username='root', password='*********')
    local_path = "testme.txt"
    remote_path = "/home/testme.txt"

    s.put(local_path, remote_path)
    s.close()

push_file_to_server()

The Error Response I am receiving is:

E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp__init__.py:61: UserWarning:
Failed to load HostKeys from C:\Users\JohnCalvin.ssh\known_hosts.
You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None). warnings.warn(wmsg, UserWarning) Traceback (most recent call last): File "E:\OneDrive\Python\GIT\DigitalCloud\pysftp_tutorial.py", line 14, in push_file_to_server() File "E:\OneDrive\Python\GIT\DigitalCloud\pysftp_tutorial.py", line 7, in push_file_to_server s = sftp.Connection(host='138.99.99.129', username='root', password='********') File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp__init__.py", line 132, in init self._tconnect['hostkey'] = self._cnopts.get_hostkey(host) File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp__init__.py", line 71, in get_hostkey raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host 138.99.99.129 found. Exception ignored in: > Traceback (most recent call last): File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp__init__.py", line 1013, in del self.close() File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp__init__.py", line 784, in close if self._sftp_live: AttributeError: 'Connection' object has no attribute '_sftp_live'

One option is to disable the host key requirement:

import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None   
with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:
    sftp.put(local_path, remote_path)

You can find more info about that here: https://stackoverflow.com/a/38355117/1060738

Important note:

By setting cnopts.hostkeys=None you'll lose the protection against Man-in-the-middle attacks by doing so. Use @martin-prikryl answer to avoid that.

Do not set cnopts.hostkeys = None (as the second most upvoted answer shows), unless you do not care about security. You lose a protection against Man-in-the-middle attacks by doing so.

Use CnOpts.hostkeys (returns HostKeys) to manage trusted host keys.

cnopts = pysftp.CnOpts(knownhosts='known_hosts')

with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:

where the known_hosts contains a server public key(s)] in a format like:

example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...

If you do not want to use an external file, you can also use

from base64 import decodebytes
# ...

keydata = b"""AAAAB3NzaC1yc2EAAAADAQAB..."""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('example.com', 'ssh-rsa', key)

with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:

Though as of pysftp 0.2.9, this approach will issue a warning, what seems like a bug:
"Failed to load HostKeys" warning while connecting to SFTP server with pysftp

An easy way to retrieve the host key in the needed format is using OpenSSH ssh-keyscan:

$ ssh-keyscan example.com
# example.com SSH-2.0-OpenSSH_5.3
example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...

(due to a bug in pysftp, this does not work, if the server uses non-standard port – the entry starts with [example.com]:port + beware of redirecting ssh-keyscan to a file in PowerShell)

You can also make the application do the same automatically:
Use Paramiko AutoAddPolicy with pysftp
(It will automatically add host keys of new hosts to known_hosts, but for known host keys, it will not accept a changed key)

Though for an absolute security, you should not retrieve the host key remotely, as you cannot be sure, if you are not being attacked already.

See my article Where do I get SSH host key fingerprint to authorize the server?
It's for my WinSCP SFTP client, but most information there is valid in general.

If you need to verify the host key using its fingerprint only, see Python - pysftp / paramiko - Verify host key using its fingerprint.

If You try to connect by pysftp to "normal" FTP You have to set hostkey to None.

import pysftp

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None 
with pysftp.Connection(host='****',username='****',password='***',port=22,cnopts=cnopts) as sftp:
    print('DO SOMETHING')

Connect to the server first with a Windows ssh client that uses the known_hosts file. PuTTy stores the data in the windows registry,however OpenSSH uses the known_hosts file, and will add entries in there after you connect. Default location for the file is %USERPROFILE%.ssh. I hope this helps

Try to use the 0.2.8 version of pysftp library. $ pip uninstall pysftp && pip install pysftp==0.2.8

And try with this:

try:
    ftp = pysftp.Connection(host, username=user, password=password)
 except:
    print("Couldn't connect to ftp")
    return False

Why this? Basically is a bug with the 0.2.9 of pysftp here all details https://github.com/Yenthe666/auto_backup/issues/47

I've implemented auto_add_key in my pysftp github fork.

auto_add_key will add the key to known_hosts if auto_add_key=True
Once a key is present for a host in known_hosts this key will be checked.

Please reffer Martin Prikryl -> answer about security concerns.

Though for an absolute security, you should not retrieve the host key remotely, as you cannot be sure, if you are not being attacked already.

import pysftp as sftp

def push_file_to_server():
    s = sftp.Connection(host='138.99.99.129', username='root', password='pass', auto_add_key=True)
    local_path = "testme.txt"
    remote_path = "/home/testme.txt"

    s.put(local_path, remote_path)
    s.close()

push_file_to_server()

Note: Why using context manager

import pysftp
with pysftp.Connection(host, username="whatever", password="whatever", auto_add_key=True) as sftp:
    #do your stuff here
#connection closed

Cook book to use different ways of pysftp.CnOpts() and hostkeys options.

Source : https://pysftp.readthedocs.io/en/release_0.2.9/cookbook.html

Host Key checking is enabled by default. It will use ~/.ssh/known_hosts by default. If you wish to disable host key checking (NOT ADVISED) you will need to modify the default CnOpts and set the .hostkeys to None.

import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
    # do stuff here

To use a completely different known_hosts file, you can override CnOpts looking for ~/.ssh/known_hosts by specifying the file when instantiating.

import pysftp
cnopts = pysftp.CnOpts(knownhosts='path/to/your/knownhostsfile')

with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
    # do stuff here

If you wish to use ~/.ssh/known_hosts but add additional known host keys you can merge with update additional known_host format files by using .load method.

import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys.load('path/to/your/extra_knownhosts')
with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
    # do stuff here

Hi We sort of had the same problem if I understand you well. So check what pysftp version you're using. If it's the latest one which is 0.2.9 downgrade to 0.2.8. Check this out. https://github.com/Yenthe666/auto_backup/issues/47

FWIR, if authentication is only username & pw, add remote server ip address to known_hosts like ssh-keyscan -H 192.168.1.162 >> ~/.ssh/known_hosts for ref https://www.techrepublic.com/article/how-to-easily-add-an-ssh-fingerprint-to-your-knownhosts-file-in-linux/