[shell] Automate scp file transfer using a shell script

I have some n number of files in a directory on my unix system. Is there a way to write a shellscript that will transfer all those files via scp to a specified remote system. I'll specify the password within the script, so that I don't have to enter it for each file.

This will work:

#!/usr/bin/expect -f

spawn scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no file1 file2 file3 user@host:/path/
expect "password:"
send "xyz123\r"
expect "*\r"
expect "\r"
interact

Try lftp

lftp -u $user,$pass sftp://$host << --EOF--

cd $directory

put $srcfile

quit

--EOF--

#!/usr/bin/expect -f
spawn scp -r BASE.zip [email protected]:/tmp
expect "password:"
send "wifinetworks\r"
expect "*\r"
expect "\r"

You can do it with ssh public/private keys only. Or use putty in which you can set the password. scp doesn't support giving password in command line.

You can find the instructions for public/private keys here: http://www.softpanorama.org/Net/Application_layer/SSH/scp.shtml

#!/usr/bin/expect -f

# connect via scp
spawn scp "[email protected]:/home/santhosh/file.dmp" /u01/dumps/file.dmp
#######################
expect {
  -re ".*es.*o.*" {
    exp_send "yes\r"
    exp_continue
  }
  -re ".*sword.*" {
    exp_send "PASSWORD\r"
  }
}
interact

http://blogs.oracle.com/SanthoshK/entry/automate_linux_scp_command

The command scp can be used like a traditional UNIX cp. SO if you do :

scp -r myDirectory/ mylogin@host:TargetDirectory

will work

you could also use rsync. It seems to work better for multiple files than scp IMHO.

rsync -avzh /path/to/dir/ user@remote:/path/to/remote/dir/

Update

You can use rsync via ssh by adding the '-e' switch:

rsync -avzh -e ssh /path/do/dir/ user@remote:/path/to/remote/dir/

why don't you try this?

password="your password"
username="username"
Ip="<IP>"
sshpass -p "$password" scp /<PATH>/final.txt $username@$Ip:/root/<PATH>

Instead of hardcoding password in a shell script, use SSH keys, its easier and secure.

$ scp -i ~/.ssh/id_rsa *.derp [email protected]:/path/to/target/directory/

assuming your private key is at ~/.ssh/id_rsa and the files you want to send can be filtered with *.derp

To generate a public / private key pair :

$ ssh-keygen -t rsa

The above will generate 2 files, ~/.ssh/id_rsa (private key) and ~/.ssh/id_rsa.pub (public key)

To setup the SSH keys for usage (one time task) : Copy the contents of ~/.ssh/id_rsa.pub and paste in a new line of ~devops/.ssh/authorized_keys in myserver.org server. If ~devops/.ssh/authorized_keys doesn't exist, feel free to create it.

A lucid how-to guide is available here.

rsync is a program that behaves in much the same way that rcp does, but has many more options and uses the rsync remote-update protocol to greatly speed up file transfers when the destination file is being updated.

The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection, using an efficient checksum-search algorithm described in the technical report that accompanies this package.

Copying folder from one location to another

   #!/usr/bin/expect -f   
   spawn rsync -a -e ssh [email protected]:/cool/cool1/* /tmp/cool/   
   expect "password:"   
   send "cool\r"   
   expect "*\r"   
   expect "\r"  

here's bash code for SCP with a .pem key file. Just save it to a script.sh file then run with 'sh script.sh'

Enjoy

#!/bin/bash
#Error function
function die(){
echo "$1"
exit 1
}

Host=ec2-53-298-45-63.us-west-1.compute.amazonaws.com
User=ubuntu
#Directory at sent destination
SendDirectory=scp
#File to send at host
FileName=filetosend.txt
#Key file
Key=MyKeyFile.pem

echo "Aperture in Process...";

#The code that will send your file scp
scp -i $Key $FileName $User@$Host:$SendDirectory || \
die "@@@@@@@Houston we have problem"

echo "########Aperture Complete#########";

If you are ok with entering your password once for every run of the script, you can do so easily using an SSH master connection.

#!/usr/bin/env bash

USER_AT_HOST="user@host"  # use "$1@$2" here if you like
SSHSOCKET=~/".ssh/$USER_AT_HOST"

# This is the only time you have to enter the password:
# Open master connection:
ssh -M -f -N -o ControlPath="$SSHSOCKET" "$USER_AT_HOST"

# These do not prompt for your password:
scp -o ControlPath="$SSHSOCKET" file1.xy "$USER_AT_HOST":remotefile1.xy
scp -o ControlPath="$SSHSOCKET" file2.xy "$USER_AT_HOST":remotefile2.xy

# You can also use the flag for normal ssh:
ssh -o ControlPath="$SSHSOCKET" "$USER_AT_HOST" "echo hello"
ssh -o ControlPath="$SSHSOCKET" "$USER_AT_HOST" "echo world"

# Close master connection:
ssh -S "$SSHSOCKET" -O exit "$USER_AT_HOST"

What about wildcards or multiple files?

scp file1 file2 more-files* user@remote:/some/dir/