What is the best method of performing an scp transfer via the Java programming language? It seems I may be able to perform this via JSSE, JSch or the bouncy castle java libraries. None of these solutions seem to have an easy answer.
This question is related to
java
scp
bouncycastle
jsse
jsch
-: Refining Fernando's answer a little, if you use Maven for dependency management :-
pom.xml:
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-jsch</artifactId>
<version>${ant-jsch.version}</version>
</dependency>
Add this dependency in your project. Latest version can be found here.
Java code:
public void scpUpload(String source, String destination) {
Scp scp = new Scp();
scp.setPort(port);
scp.setLocalFile(source);
scp.setTodir(username + ":" + password + "@" + host + ":" + destination);
scp.setProject(new Project());
scp.setTrust(true);
scp.execute();
}
I looked at a lot of these solutions and didn't like many of them. Mostly because the annoying step of having to identify your known hosts. That and JSCH is at a ridiculously low level relative to the scp command.
I found a library that doesn't require this but it's bundled up and used as a command line tool. https://code.google.com/p/scp-java-client/
I looked through the source code and discovered how to use it without the command line. Here's an example of uploading:
uk.co.marcoratto.scp.SCP scp = new uk.co.marcoratto.scp.SCP(new uk.co.marcoratto.scp.listeners.SCPListenerPrintStream());
scp.setUsername("root");
scp.setPassword("blah");
scp.setTrust(true);
scp.setFromUri(file.getAbsolutePath());
scp.setToUri("root@host:/path/on/remote");
scp.execute();
The biggest downside is that it's not in a maven repo (that I could find). But, the ease of use is worth it to me.
jsCH has worked great for me. Below is an example of a method that will connect to sftp server and download files to specified directory. It is recommended to stay away from disabling StrictHostKeyChecking. Although a little bit more difficult to set up, for security reasons specifying the known hosts should be the norm.
jsch.setKnownHosts("C:\Users\test\known_hosts"); recommended
JSch.setConfig("StrictHostKeyChecking", "no"); - not recommended
import com.jcraft.jsch.*;
public void downloadFtp(String userName, String password, String host, int port, String path) {
Session session = null;
Channel channel = null;
try {
JSch ssh = new JSch();
JSch.setConfig("StrictHostKeyChecking", "no");
session = ssh.getSession(userName, host, port);
session.setPassword(password);
session.connect();
channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftp = (ChannelSftp) channel;
sftp.get(path, "specify path to where you want the files to be output");
} catch (JSchException e) {
System.out.println(userName);
e.printStackTrace();
} catch (SftpException e) {
System.out.println(userName);
e.printStackTrace();
} finally {
if (channel != null) {
channel.disconnect();
}
if (session != null) {
session.disconnect();
}
}
}
This is high-level solution, no need to reinvent. Quick and dirty!
1) First, go to http://ant.apache.org/bindownload.cgi and download latest Apache Ant binary. (nowadays, apache-ant-1.9.4-bin.zip).
2) Extract the downloaded file and find the JAR ant-jsch.jar ("apache-ant-1.9.4/lib/ant-jsch.jar"). Add this JAR in your project. Also ant-launcher.jar and ant.jar.
3) Go to Jcraft jsch SouceForge Project and download the jar. Nowadays, jsch-0.1.52.jar. Also Add this JAR in your project.
Now, can you easyly use into java code the Ant Classes Scp for copy files over network or SSHExec for commands in SSH servers.
4) Code Example Scp:
// This make scp copy of
// one local file to remote dir
org.apache.tools.ant.taskdefs.optional.ssh.Scp scp = new Scp();
int portSSH = 22;
String srvrSSH = "ssh.your.domain";
String userSSH = "anyuser";
String pswdSSH = new String ( jPasswordField1.getPassword() );
String localFile = "C:\\localfile.txt";
String remoteDir = "/uploads/";
scp.setPort( portSSH );
scp.setLocalFile( localFile );
scp.setTodir( userSSH + ":" + pswdSSH + "@" + srvrSSH + ":" + remoteDir );
scp.setProject( new Project() );
scp.setTrust( true );
scp.execute();
-: Refining Fernando's answer a little, if you use Maven for dependency management :-
pom.xml:
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-jsch</artifactId>
<version>${ant-jsch.version}</version>
</dependency>
Add this dependency in your project. Latest version can be found here.
Java code:
public void scpUpload(String source, String destination) {
Scp scp = new Scp();
scp.setPort(port);
scp.setLocalFile(source);
scp.setTodir(username + ":" + password + "@" + host + ":" + destination);
scp.setProject(new Project());
scp.setTrust(true);
scp.execute();
}
This is high-level solution, no need to reinvent. Quick and dirty!
1) First, go to http://ant.apache.org/bindownload.cgi and download latest Apache Ant binary. (nowadays, apache-ant-1.9.4-bin.zip).
2) Extract the downloaded file and find the JAR ant-jsch.jar ("apache-ant-1.9.4/lib/ant-jsch.jar"). Add this JAR in your project. Also ant-launcher.jar and ant.jar.
3) Go to Jcraft jsch SouceForge Project and download the jar. Nowadays, jsch-0.1.52.jar. Also Add this JAR in your project.
Now, can you easyly use into java code the Ant Classes Scp for copy files over network or SSHExec for commands in SSH servers.
4) Code Example Scp:
// This make scp copy of
// one local file to remote dir
org.apache.tools.ant.taskdefs.optional.ssh.Scp scp = new Scp();
int portSSH = 22;
String srvrSSH = "ssh.your.domain";
String userSSH = "anyuser";
String pswdSSH = new String ( jPasswordField1.getPassword() );
String localFile = "C:\\localfile.txt";
String remoteDir = "/uploads/";
scp.setPort( portSSH );
scp.setLocalFile( localFile );
scp.setTodir( userSSH + ":" + pswdSSH + "@" + srvrSSH + ":" + remoteDir );
scp.setProject( new Project() );
scp.setTrust( true );
scp.execute();
I need to copy folder recursively, after trying different solutions, finally end up by ProcessBuilder + expect/spawn
scpFile("192.168.1.1", "root","password","/tmp/1","/tmp");
public void scpFile(String host, String username, String password, String src, String dest) throws Exception {
String[] scpCmd = new String[]{"expect", "-c", String.format("spawn scp -r %s %s@%s:%s\n", src, username, host, dest) +
"expect \"?assword:\"\n" +
String.format("send \"%s\\r\"\n", password) +
"expect eof"};
ProcessBuilder pb = new ProcessBuilder(scpCmd);
System.out.println("Run shell command: " + Arrays.toString(scpCmd));
Process process = pb.start();
int errCode = process.waitFor();
System.out.println("Echo command executed, any errors? " + (errCode == 0 ? "No" : "Yes"));
System.out.println("Echo Output:\n" + output(process.getInputStream()));
if(errCode != 0) throw new Exception();
}
The openssh project lists several Java alternatives, Trilead SSH for Java seems to fit what you're asking for.
I wrapped Jsch with some utility methods to make it a bit friendlier and called it
Available here: https://github.com/willwarren/jscp
SCP utility to tar a folder, zip it, and scp it somewhere, then unzip it.
Usage:
// create secure context
SecureContext context = new SecureContext("userName", "localhost");
// set optional security configurations.
context.setTrustAllHosts(true);
context.setPrivateKeyFile(new File("private/key"));
// Console requires JDK 1.7
// System.out.println("enter password:");
// context.setPassword(System.console().readPassword());
Jscp.exec(context,
"src/dir",
"destination/path",
// regex ignore list
Arrays.asList("logs/log[0-9]*.txt",
"backups")
);
Also includes useful classes - Scp and Exec, and a TarAndGzip, which work in pretty much the same way.
Take a look here
That is the source code for Ants' SCP task. The code in the "execute" method is where the nuts and bolts of it are. This should give you a fair idea of what is required. It uses JSch i believe.
Alternatively you could also directly execute this Ant task from your java code.
I wrote an scp server which is much easier than others. I use Apache MINA project (Apache SSHD) to develop it. You can take a look here: https://github.com/boomz/JSCP
Also you can download the jar file from /jar directory.
How to use? Take a look on: https://github.com/boomz/JSCP/blob/master/src/Main.java
Like some here, I ended up writing a wrapper around the JSch library.
It's called way-secshell and it is hosted on GitHub:
https://github.com/objectos/way-secshell
// scp myfile.txt localhost:/tmp
File file = new File("myfile.txt");
Scp res = WaySSH.scp()
.file(file)
.toHost("localhost")
.at("/tmp")
.send();
I use this SFTP API which has SCP called Zehon, it's great, so easy to use with a lot of sample code. Here is the site http://www.zehon.com
I looked at a lot of these solutions and didn't like many of them. Mostly because the annoying step of having to identify your known hosts. That and JSCH is at a ridiculously low level relative to the scp command.
I found a library that doesn't require this but it's bundled up and used as a command line tool. https://code.google.com/p/scp-java-client/
I looked through the source code and discovered how to use it without the command line. Here's an example of uploading:
uk.co.marcoratto.scp.SCP scp = new uk.co.marcoratto.scp.SCP(new uk.co.marcoratto.scp.listeners.SCPListenerPrintStream());
scp.setUsername("root");
scp.setPassword("blah");
scp.setTrust(true);
scp.setFromUri(file.getAbsolutePath());
scp.setToUri("root@host:/path/on/remote");
scp.execute();
The biggest downside is that it's not in a maven repo (that I could find). But, the ease of use is worth it to me.
Here is an example to upload a file using JSch:
ScpUploader.java:
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;
import java.io.ByteArrayInputStream;
import java.util.Properties;
public final class ScpUploader
{
public static ScpUploader newInstance()
{
return new ScpUploader();
}
private volatile Session session;
private volatile ChannelSftp channel;
private ScpUploader(){}
public synchronized void connect(String host, int port, String username, String password) throws JSchException
{
JSch jsch = new JSch();
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session = jsch.getSession(username, host, port);
session.setPassword(password);
session.setConfig(config);
session.setInputStream(System.in);
session.connect();
channel = (ChannelSftp) session.openChannel("sftp");
channel.connect();
}
public synchronized void uploadFile(String directoryPath, String fileName, byte[] fileBytes, boolean overwrite) throws SftpException
{
if(session == null || channel == null)
{
System.err.println("No open session!");
return;
}
// a workaround to check if the directory exists. Otherwise, create it
channel.cd("/");
String[] directories = directoryPath.split("/");
for(String directory : directories)
{
if(directory.length() > 0)
{
try
{
channel.cd(directory);
}
catch(SftpException e)
{
// swallowed exception
System.out.println("The directory (" + directory + ") seems to be not exist. We will try to create it.");
try
{
channel.mkdir(directory);
channel.cd(directory);
System.out.println("The directory (" + directory + ") is created successfully!");
}
catch(SftpException e1)
{
System.err.println("The directory (" + directory + ") is failed to be created!");
e1.printStackTrace();
return;
}
}
}
}
channel.put(new ByteArrayInputStream(fileBytes), directoryPath + "/" + fileName, overwrite ? ChannelSftp.OVERWRITE : ChannelSftp.RESUME);
}
public synchronized void disconnect()
{
if(session == null || channel == null)
{
System.err.println("No open session!");
return;
}
channel.exit();
channel.disconnect();
session.disconnect();
channel = null;
session = null;
}
}
AppEntryPoint.java:
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.SftpException;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
public final class AppEntryPoint
{
private static final String HOST = "192.168.1.1";
private static final int PORT = 22;
private static final String USERNAME = "root";
private static final String PASSWORD = "root";
public static void main(String[] args) throws IOException
{
ScpUploader scpUploader = ScpUploader.newInstance();
try
{
scpUploader.connect(HOST, PORT, USERNAME, PASSWORD);
}
catch(JSchException e)
{
System.err.println("Failed to connect the server!");
e.printStackTrace();
return;
}
System.out.println("Successfully connected to the server!");
byte[] fileBytes = Files.readAllBytes(Paths.get("C:/file.zip"));
try
{
scpUploader.uploadFile("/test/files", "file.zip", fileBytes, true); // if overwrite == false, it won't throw exception if the file exists
System.out.println("Successfully uploaded the file!");
}
catch(SftpException e)
{
System.err.println("Failed to upload the file!");
e.printStackTrace();
}
scpUploader.disconnect();
}
}
The openssh project lists several Java alternatives, Trilead SSH for Java seems to fit what you're asking for.
Take a look here
That is the source code for Ants' SCP task. The code in the "execute" method is where the nuts and bolts of it are. This should give you a fair idea of what is required. It uses JSch i believe.
Alternatively you could also directly execute this Ant task from your java code.
JSch is a nice library to work with. It has quite an easy answer for your question.
JSch jsch=new JSch();
Session session=jsch.getSession(user, host, 22);
session.setPassword("password");
Properties config = new Properties();
config.put("StrictHostKeyChecking","no");
session.setConfig(config);
session.connect();
boolean ptimestamp = true;
// exec 'scp -t rfile' remotely
String command="scp " + (ptimestamp ? "-p" :"") +" -t "+rfile;
Channel channel=session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
// get I/O streams for remote scp
OutputStream out=channel.getOutputStream();
InputStream in=channel.getInputStream();
channel.connect();
if(checkAck(in)!=0){
System.exit(0);
}
File _lfile = new File(lfile);
if(ptimestamp){
command="T "+(_lfile.lastModified()/1000)+" 0";
// The access time should be sent here,
// but it is not accessible with JavaAPI ;-<
command+=(" "+(_lfile.lastModified()/1000)+" 0\n");
out.write(command.getBytes()); out.flush();
if(checkAck(in)!=0){
System.exit(0);
}
}
You can find complete code at
http://faisalbhagat.blogspot.com/2013/09/java-uploading-file-remotely-via-scp.html
Like some here, I ended up writing a wrapper around the JSch library.
It's called way-secshell and it is hosted on GitHub:
https://github.com/objectos/way-secshell
// scp myfile.txt localhost:/tmp
File file = new File("myfile.txt");
Scp res = WaySSH.scp()
.file(file)
.toHost("localhost")
.at("/tmp")
.send();
JSch is a nice library to work with. It has quite an easy answer for your question.
JSch jsch=new JSch();
Session session=jsch.getSession(user, host, 22);
session.setPassword("password");
Properties config = new Properties();
config.put("StrictHostKeyChecking","no");
session.setConfig(config);
session.connect();
boolean ptimestamp = true;
// exec 'scp -t rfile' remotely
String command="scp " + (ptimestamp ? "-p" :"") +" -t "+rfile;
Channel channel=session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
// get I/O streams for remote scp
OutputStream out=channel.getOutputStream();
InputStream in=channel.getInputStream();
channel.connect();
if(checkAck(in)!=0){
System.exit(0);
}
File _lfile = new File(lfile);
if(ptimestamp){
command="T "+(_lfile.lastModified()/1000)+" 0";
// The access time should be sent here,
// but it is not accessible with JavaAPI ;-<
command+=(" "+(_lfile.lastModified()/1000)+" 0\n");
out.write(command.getBytes()); out.flush();
if(checkAck(in)!=0){
System.exit(0);
}
}
You can find complete code at
http://faisalbhagat.blogspot.com/2013/09/java-uploading-file-remotely-via-scp.html
Take a look here
That is the source code for Ants' SCP task. The code in the "execute" method is where the nuts and bolts of it are. This should give you a fair idea of what is required. It uses JSch i believe.
Alternatively you could also directly execute this Ant task from your java code.
I need to copy folder recursively, after trying different solutions, finally end up by ProcessBuilder + expect/spawn
scpFile("192.168.1.1", "root","password","/tmp/1","/tmp");
public void scpFile(String host, String username, String password, String src, String dest) throws Exception {
String[] scpCmd = new String[]{"expect", "-c", String.format("spawn scp -r %s %s@%s:%s\n", src, username, host, dest) +
"expect \"?assword:\"\n" +
String.format("send \"%s\\r\"\n", password) +
"expect eof"};
ProcessBuilder pb = new ProcessBuilder(scpCmd);
System.out.println("Run shell command: " + Arrays.toString(scpCmd));
Process process = pb.start();
int errCode = process.waitFor();
System.out.println("Echo command executed, any errors? " + (errCode == 0 ? "No" : "Yes"));
System.out.println("Echo Output:\n" + output(process.getInputStream()));
if(errCode != 0) throw new Exception();
}
I wrote an scp server which is much easier than others. I use Apache MINA project (Apache SSHD) to develop it. You can take a look here: https://github.com/boomz/JSCP
Also you can download the jar file from /jar directory.
How to use? Take a look on: https://github.com/boomz/JSCP/blob/master/src/Main.java
Take a look here
That is the source code for Ants' SCP task. The code in the "execute" method is where the nuts and bolts of it are. This should give you a fair idea of what is required. It uses JSch i believe.
Alternatively you could also directly execute this Ant task from your java code.
I wrapped Jsch with some utility methods to make it a bit friendlier and called it
Available here: https://github.com/willwarren/jscp
SCP utility to tar a folder, zip it, and scp it somewhere, then unzip it.
Usage:
// create secure context
SecureContext context = new SecureContext("userName", "localhost");
// set optional security configurations.
context.setTrustAllHosts(true);
context.setPrivateKeyFile(new File("private/key"));
// Console requires JDK 1.7
// System.out.println("enter password:");
// context.setPassword(System.console().readPassword());
Jscp.exec(context,
"src/dir",
"destination/path",
// regex ignore list
Arrays.asList("logs/log[0-9]*.txt",
"backups")
);
Also includes useful classes - Scp and Exec, and a TarAndGzip, which work in pretty much the same way.
jsCH has worked great for me. Below is an example of a method that will connect to sftp server and download files to specified directory. It is recommended to stay away from disabling StrictHostKeyChecking. Although a little bit more difficult to set up, for security reasons specifying the known hosts should be the norm.
jsch.setKnownHosts("C:\Users\test\known_hosts"); recommended
JSch.setConfig("StrictHostKeyChecking", "no"); - not recommended
import com.jcraft.jsch.*;
public void downloadFtp(String userName, String password, String host, int port, String path) {
Session session = null;
Channel channel = null;
try {
JSch ssh = new JSch();
JSch.setConfig("StrictHostKeyChecking", "no");
session = ssh.getSession(userName, host, port);
session.setPassword(password);
session.connect();
channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftp = (ChannelSftp) channel;
sftp.get(path, "specify path to where you want the files to be output");
} catch (JSchException e) {
System.out.println(userName);
e.printStackTrace();
} catch (SftpException e) {
System.out.println(userName);
e.printStackTrace();
} finally {
if (channel != null) {
channel.disconnect();
}
if (session != null) {
session.disconnect();
}
}
}
The openssh project lists several Java alternatives, Trilead SSH for Java seems to fit what you're asking for.
Here is an example to upload a file using JSch:
ScpUploader.java:
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.SftpException;
import java.io.ByteArrayInputStream;
import java.util.Properties;
public final class ScpUploader
{
public static ScpUploader newInstance()
{
return new ScpUploader();
}
private volatile Session session;
private volatile ChannelSftp channel;
private ScpUploader(){}
public synchronized void connect(String host, int port, String username, String password) throws JSchException
{
JSch jsch = new JSch();
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session = jsch.getSession(username, host, port);
session.setPassword(password);
session.setConfig(config);
session.setInputStream(System.in);
session.connect();
channel = (ChannelSftp) session.openChannel("sftp");
channel.connect();
}
public synchronized void uploadFile(String directoryPath, String fileName, byte[] fileBytes, boolean overwrite) throws SftpException
{
if(session == null || channel == null)
{
System.err.println("No open session!");
return;
}
// a workaround to check if the directory exists. Otherwise, create it
channel.cd("/");
String[] directories = directoryPath.split("/");
for(String directory : directories)
{
if(directory.length() > 0)
{
try
{
channel.cd(directory);
}
catch(SftpException e)
{
// swallowed exception
System.out.println("The directory (" + directory + ") seems to be not exist. We will try to create it.");
try
{
channel.mkdir(directory);
channel.cd(directory);
System.out.println("The directory (" + directory + ") is created successfully!");
}
catch(SftpException e1)
{
System.err.println("The directory (" + directory + ") is failed to be created!");
e1.printStackTrace();
return;
}
}
}
}
channel.put(new ByteArrayInputStream(fileBytes), directoryPath + "/" + fileName, overwrite ? ChannelSftp.OVERWRITE : ChannelSftp.RESUME);
}
public synchronized void disconnect()
{
if(session == null || channel == null)
{
System.err.println("No open session!");
return;
}
channel.exit();
channel.disconnect();
session.disconnect();
channel = null;
session = null;
}
}
AppEntryPoint.java:
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.SftpException;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
public final class AppEntryPoint
{
private static final String HOST = "192.168.1.1";
private static final int PORT = 22;
private static final String USERNAME = "root";
private static final String PASSWORD = "root";
public static void main(String[] args) throws IOException
{
ScpUploader scpUploader = ScpUploader.newInstance();
try
{
scpUploader.connect(HOST, PORT, USERNAME, PASSWORD);
}
catch(JSchException e)
{
System.err.println("Failed to connect the server!");
e.printStackTrace();
return;
}
System.out.println("Successfully connected to the server!");
byte[] fileBytes = Files.readAllBytes(Paths.get("C:/file.zip"));
try
{
scpUploader.uploadFile("/test/files", "file.zip", fileBytes, true); // if overwrite == false, it won't throw exception if the file exists
System.out.println("Successfully uploaded the file!");
}
catch(SftpException e)
{
System.err.println("Failed to upload the file!");
e.printStackTrace();
}
scpUploader.disconnect();
}
}
I use this SFTP API which has SCP called Zehon, it's great, so easy to use with a lot of sample code. Here is the site http://www.zehon.com
Source: Stackoverflow.com