How to copy a file to a remote server in Python using SCP or SSH

Question

I have a text file on my local machine that is generated by a daily Python script run in cron    I would like to add a bit of code to have that file sent securely to my server over SSH

User · Answer

A very simple approach is the following    import os os system  sshpass -p  password  scp user host  path to file        No python library are required  only os   and it works  however using this method relies on another ssh client to be installed  This could result in undesired behavior if ran on another system

User · Answer

You can do something like this  to handle the host key checking as well  import os os system  sshpass -p password scp -o StrictHostKeyChecking no local file path username hostname remote path

User · Answer

fabric could be used to upload files vis ssh      usr bin env python from fabric api import execute  put from fabric network import disconnect all  if   name       main         import sys       specify hostname to connect to and the remote local paths     srcdir  remote dirname  hostname   sys argv 1       try          s   execute put  srcdir  remote dirname  host hostname          print repr s       finally          disconnect all

User · Answer

You can call the scp bash command  it copies files over SSH  with subprocess run   import subprocess subprocess run   scp   FILE   USER SERVER PATH     e g  subprocess run   scp    foo bar    joe srvr net  path to foo bar      If you re creating the file that you want to send in the same Python program  you ll want to call subprocess run command outside the with block you re using to open the file  or call  close   on the file first if you re not using a with block   so you know it s flushed to disk from Python   You need to generate  on the source machine  and install  on the destination machine  an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key  in other words  so your script doesn t ask for a password

User · Answer

There are a couple of different ways to approach the problem    Wrap command-line programs use a Python library that provides SSH capabilities  eg - Paramiko or Twisted Conch    Each approach has its own quirks  You will need to setup SSH keys to enable password-less logins if you are wrapping system commands like  ssh    scp  or  rsync   You can embed a password in a script using Paramiko or some other library  but you might find the lack of documentation frustrating  especially if you are not familiar with the basics of the SSH connection  eg - key exchanges  agents  etc   It probably goes without saying that SSH keys are almost always a better idea than passwords for this sort of stuff   NOTE  its hard to beat rsync if you plan on transferring files via SSH  especially if the alternative is plain old scp   I ve used Paramiko with an eye towards replacing system calls but found myself drawn back to the wrapped commands due to their ease of use and immediate familiarity  You might be different  I gave Conch the once-over some time ago but it didn t appeal to me   If opting for the system-call path  Python offers an array of options such as os system or the commands subprocess modules  I d go with the subprocess module if using version 2 4

User · Answer

You can call the scp bash command  it copies files over SSH  with subprocess run   import subprocess subprocess run   scp   FILE   USER SERVER PATH     e g  subprocess run   scp    foo bar    joe srvr net  path to foo bar      If you re creating the file that you want to send in the same Python program  you ll want to call subprocess run command outside the with block you re using to open the file  or call  close   on the file first if you re not using a with block   so you know it s flushed to disk from Python   You need to generate  on the source machine  and install  on the destination machine  an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key  in other words  so your script doesn t ask for a password

User · Answer

There are a couple of different ways to approach the problem    Wrap command-line programs use a Python library that provides SSH capabilities  eg - Paramiko or Twisted Conch    Each approach has its own quirks  You will need to setup SSH keys to enable password-less logins if you are wrapping system commands like  ssh    scp  or  rsync   You can embed a password in a script using Paramiko or some other library  but you might find the lack of documentation frustrating  especially if you are not familiar with the basics of the SSH connection  eg - key exchanges  agents  etc   It probably goes without saying that SSH keys are almost always a better idea than passwords for this sort of stuff   NOTE  its hard to beat rsync if you plan on transferring files via SSH  especially if the alternative is plain old scp   I ve used Paramiko with an eye towards replacing system calls but found myself drawn back to the wrapped commands due to their ease of use and immediate familiarity  You might be different  I gave Conch the once-over some time ago but it didn t appeal to me   If opting for the system-call path  Python offers an array of options such as os system or the commands subprocess modules  I d go with the subprocess module if using version 2 4

User · Answer

You d probably use the subprocess module  Something like this   import subprocess p   subprocess Popen   scp   myfile  destination   sts   os waitpid p pid  0    Where destination is probably of the form user remotehost remotepath  Thanks to  Charles Duffy for pointing out the weakness in my original answer  which used a single string argument to specify the scp operation shell True - that wouldn t handle whitespace in paths   The module documentation has examples of error checking that you may want to perform in conjunction with this operation   Ensure that you ve set up proper credentials so that you can perform an unattended  passwordless scp between the machines  There is a stackoverflow question for this already

User · Answer

To do this in Python  i e  not wrapping scp through subprocess Popen or similar  with the Paramiko library  you would do something like this   import os import paramiko  ssh   paramiko SSHClient    ssh load host keys os path expanduser os path join        ssh    known hosts     ssh connect server  username username  password password  sftp   ssh open sftp   sftp put localpath  remotepath  sftp close   ssh close      You would probably want to deal with unknown hosts  errors  creating any directories necessary  and so on

User · Answer

Reached the same problem  but instead of  hacking  or emulating command line   Found this answer here   from paramiko import SSHClient from scp import SCPClient  ssh   SSHClient   ssh load system host keys   ssh connect  example com    with SCPClient ssh get transport    as scp      scp put  test txt    test2 txt       scp get  test2 txt

User · Answer

You d probably use the subprocess module  Something like this   import subprocess p   subprocess Popen   scp   myfile  destination   sts   os waitpid p pid  0    Where destination is probably of the form user remotehost remotepath  Thanks to  Charles Duffy for pointing out the weakness in my original answer  which used a single string argument to specify the scp operation shell True - that wouldn t handle whitespace in paths   The module documentation has examples of error checking that you may want to perform in conjunction with this operation   Ensure that you ve set up proper credentials so that you can perform an unattended  passwordless scp between the machines  There is a stackoverflow question for this already

User · Answer

You can do something like this  to handle the host key checking as well  import os os system  sshpass -p password scp -o StrictHostKeyChecking no local file path username hostname remote path

User · Answer

Reached the same problem  but instead of  hacking  or emulating command line   Found this answer here   from paramiko import SSHClient from scp import SCPClient  ssh   SSHClient   ssh load system host keys   ssh connect  example com    with SCPClient ssh get transport    as scp      scp put  test txt    test2 txt       scp get  test2 txt

User · Answer

Using the external resource paramiko       from paramiko import SSHClient     from scp import SCPClient     import os      ssh   SSHClient        ssh load host keys os path expanduser os path join        ssh    known hosts         ssh connect server  username  username   password  password       with SCPClient ssh get transport    as scp              scp put  test txt    test2 txt

User · Answer

You can use the vassal package  which is exactly designed for this   All you need is to install vassal and do  from vassal terminal import Terminal shell   Terminal   scp username host  home foo txt foo local txt    shell run     Also  it will save you authenticate credential and don t need to type them again and again

User · Answer

Kind of hacky  but the following should work     import os filePath     foo bar baz py  serverPath     blah boo boom py  os system  scp   filePath   user myserver com   serverPath

User · Answer

Try this if you wan t to use SSL certificates   import subprocess  try        Set scp and ssh data      connUser    john      connHost    my host com      connPath     home john       connPrivateKey     home user myKey pem         Use scp to send file from local to host      scp   subprocess Popen   scp    -i   connPrivateKey   myFile txt              format connUser  connHost  connPath     except CalledProcessError      print  ERROR  Connection to host failed

User · Answer

You can call the scp bash command  it copies files over SSH  with subprocess run   import subprocess subprocess run   scp   FILE   USER SERVER PATH     e g  subprocess run   scp    foo bar    joe srvr net  path to foo bar      If you re creating the file that you want to send in the same Python program  you ll want to call subprocess run command outside the with block you re using to open the file  or call  close   on the file first if you re not using a with block   so you know it s flushed to disk from Python   You need to generate  on the source machine  and install  on the destination machine  an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key  in other words  so your script doesn t ask for a password

User · Answer

Calling scp command via subprocess doesn t allow to receive the progress report inside the script  pexpect could be used to extract that info   import pipes import re import pexpect     pip install pexpect  def progress locals         extract percents     print int re search br   d       locals  child   after  group 1     command    scp  s  s    tuple map pipes quote   srcfile  destination    pexpect run command  events  r  d     progress     See python copy file in local network  linux -  linux

User · Answer

There are a couple of different ways to approach the problem    Wrap command-line programs use a Python library that provides SSH capabilities  eg - Paramiko or Twisted Conch    Each approach has its own quirks  You will need to setup SSH keys to enable password-less logins if you are wrapping system commands like  ssh    scp  or  rsync   You can embed a password in a script using Paramiko or some other library  but you might find the lack of documentation frustrating  especially if you are not familiar with the basics of the SSH connection  eg - key exchanges  agents  etc   It probably goes without saying that SSH keys are almost always a better idea than passwords for this sort of stuff   NOTE  its hard to beat rsync if you plan on transferring files via SSH  especially if the alternative is plain old scp   I ve used Paramiko with an eye towards replacing system calls but found myself drawn back to the wrapped commands due to their ease of use and immediate familiarity  You might be different  I gave Conch the once-over some time ago but it didn t appeal to me   If opting for the system-call path  Python offers an array of options such as os system or the commands subprocess modules  I d go with the subprocess module if using version 2 4

User · Answer

Using the external resource paramiko       from paramiko import SSHClient     from scp import SCPClient     import os      ssh   SSHClient        ssh load host keys os path expanduser os path join        ssh    known hosts         ssh connect server  username  username   password  password       with SCPClient ssh get transport    as scp              scp put  test txt    test2 txt

User · Answer

fabric could be used to upload files vis ssh      usr bin env python from fabric api import execute  put from fabric network import disconnect all  if   name       main         import sys       specify hostname to connect to and the remote local paths     srcdir  remote dirname  hostname   sys argv 1       try          s   execute put  srcdir  remote dirname  host hostname          print repr s       finally          disconnect all

User · Answer

Kind of hacky  but the following should work     import os filePath     foo bar baz py  serverPath     blah boo boom py  os system  scp   filePath   user myserver com   serverPath

User · Answer

You d probably use the subprocess module  Something like this   import subprocess p   subprocess Popen   scp   myfile  destination   sts   os waitpid p pid  0    Where destination is probably of the form user remotehost remotepath  Thanks to  Charles Duffy for pointing out the weakness in my original answer  which used a single string argument to specify the scp operation shell True - that wouldn t handle whitespace in paths   The module documentation has examples of error checking that you may want to perform in conjunction with this operation   Ensure that you ve set up proper credentials so that you can perform an unattended  passwordless scp between the machines  There is a stackoverflow question for this already

User · Answer

Calling scp command via subprocess doesn t allow to receive the progress report inside the script  pexpect could be used to extract that info   import pipes import re import pexpect     pip install pexpect  def progress locals         extract percents     print int re search br   d       locals  child   after  group 1     command    scp  s  s    tuple map pipes quote   srcfile  destination    pexpect run command  events  r  d     progress     See python copy file in local network  linux -  linux

User · Answer

I used sshfs to mount the remote directory via ssh  and shutil to copy the files     mkdir   sshmount   sshfs user remotehost  path to remote dst   sshmount   Then in python   import shutil shutil copy  a txt      sshmount     This method has the advantage that you can stream data over if you are generating data rather than caching locally and sending a single large file

User · Answer

I used sshfs to mount the remote directory via ssh  and shutil to copy the files     mkdir   sshmount   sshfs user remotehost  path to remote dst   sshmount   Then in python   import shutil shutil copy  a txt      sshmount     This method has the advantage that you can stream data over if you are generating data rather than caching locally and sending a single large file

User · Answer

A very simple approach is the following    import os os system  sshpass -p  password  scp user host  path to file        No python library are required  only os   and it works  however using this method relies on another ssh client to be installed  This could result in undesired behavior if ran on another system

User · Answer

You d probably use the subprocess module  Something like this   import subprocess p   subprocess Popen   scp   myfile  destination   sts   os waitpid p pid  0    Where destination is probably of the form user remotehost remotepath  Thanks to  Charles Duffy for pointing out the weakness in my original answer  which used a single string argument to specify the scp operation shell True - that wouldn t handle whitespace in paths   The module documentation has examples of error checking that you may want to perform in conjunction with this operation   Ensure that you ve set up proper credentials so that you can perform an unattended  passwordless scp between the machines  There is a stackoverflow question for this already

User · Answer

To do this in Python  i e  not wrapping scp through subprocess Popen or similar  with the Paramiko library  you would do something like this   import os import paramiko  ssh   paramiko SSHClient    ssh load host keys os path expanduser os path join        ssh    known hosts     ssh connect server  username username  password password  sftp   ssh open sftp   sftp put localpath  remotepath  sftp close   ssh close      You would probably want to deal with unknown hosts  errors  creating any directories necessary  and so on

User · Answer

Kind of hacky  but the following should work     import os filePath     foo bar baz py  serverPath     blah boo boom py  os system  scp   filePath   user myserver com   serverPath

User · Answer

Kind of hacky  but the following should work     import os filePath     foo bar baz py  serverPath     blah boo boom py  os system  scp   filePath   user myserver com   serverPath

User · Answer

There are a couple of different ways to approach the problem    Wrap command-line programs use a Python library that provides SSH capabilities  eg - Paramiko or Twisted Conch    Each approach has its own quirks  You will need to setup SSH keys to enable password-less logins if you are wrapping system commands like  ssh    scp  or  rsync   You can embed a password in a script using Paramiko or some other library  but you might find the lack of documentation frustrating  especially if you are not familiar with the basics of the SSH connection  eg - key exchanges  agents  etc   It probably goes without saying that SSH keys are almost always a better idea than passwords for this sort of stuff   NOTE  its hard to beat rsync if you plan on transferring files via SSH  especially if the alternative is plain old scp   I ve used Paramiko with an eye towards replacing system calls but found myself drawn back to the wrapped commands due to their ease of use and immediate familiarity  You might be different  I gave Conch the once-over some time ago but it didn t appeal to me   If opting for the system-call path  Python offers an array of options such as os system or the commands subprocess modules  I d go with the subprocess module if using version 2 4

User · Answer

Try this if you wan t to use SSL certificates   import subprocess  try        Set scp and ssh data      connUser    john      connHost    my host com      connPath     home john       connPrivateKey     home user myKey pem         Use scp to send file from local to host      scp   subprocess Popen   scp    -i   connPrivateKey   myFile txt              format connUser  connHost  connPath     except CalledProcessError      print  ERROR  Connection to host failed

User · Answer

You can use the vassal package  which is exactly designed for this   All you need is to install vassal and do  from vassal terminal import Terminal shell   Terminal   scp username host  home foo txt foo local txt    shell run     Also  it will save you authenticate credential and don t need to type them again and again

User · Answer

To do this in Python  i e  not wrapping scp through subprocess Popen or similar  with the Paramiko library  you would do something like this   import os import paramiko  ssh   paramiko SSHClient    ssh load host keys os path expanduser os path join        ssh    known hosts     ssh connect server  username username  password password  sftp   ssh open sftp   sftp put localpath  remotepath  sftp close   ssh close      You would probably want to deal with unknown hosts  errors  creating any directories necessary  and so on

User · Answer

You can call the scp bash command  it copies files over SSH  with subprocess run   import subprocess subprocess run   scp   FILE   USER SERVER PATH     e g  subprocess run   scp    foo bar    joe srvr net  path to foo bar      If you re creating the file that you want to send in the same Python program  you ll want to call subprocess run command outside the with block you re using to open the file  or call  close   on the file first if you re not using a with block   so you know it s flushed to disk from Python   You need to generate  on the source machine  and install  on the destination machine  an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key  in other words  so your script doesn t ask for a password

User · Answer

To do this in Python  i e  not wrapping scp through subprocess Popen or similar  with the Paramiko library  you would do something like this   import os import paramiko  ssh   paramiko SSHClient    ssh load host keys os path expanduser os path join        ssh    known hosts     ssh connect server  username username  password password  sftp   ssh open sftp   sftp put localpath  remotepath  sftp close   ssh close      You would probably want to deal with unknown hosts  errors  creating any directories necessary  and so on

[python] How to copy a file to a remote server in Python using SCP or SSH?

Examples related to python

Examples related to ssh

Examples related to automation

Examples related to scp