[maven] Best practices for copying files with Maven

I have config files and various documents that I want to copy from the dev environment to the dev-server directory using Maven2. Strangely, Maven does not seem strong at this task.

Some of the options:

  • Simple use a copy task in Maven
<copy file="src/main/resources/config.properties" tofile="${project.server.config}/config.properties"/>
  • Use the Ant plugin to execute copy from Ant.

    • Construct an artifact of type zip, alongside the "main" artifact of the POM which is usually of type jar, then unpack that artifact from the repository into the target directory.

    • maven-resources plugin, as mentioned below.

    • Maven Assembly plugin -- but this seems to require a lot of manual definitions, when I want to do things simply and "conventionally."

    • This page even shows how to build a plugin to do copying!

    • maven-upload plugin, as mentioned below.

    • maven-dependency-plugin with copy, as mentioned below.


All these seem needlessly ad hoc: Maven is supposed to excel at doing these standard tasks without fuss and bother.

Any advice?

This question is related to maven maven-2 build-process release-management

The answer is


If someone wants total control over the path of the source and destination paths, then using maven-antrun-plugin's copy task is the best option. This approach will allow you to copy between any paths on the system, irrespective of the concerned paths being within the mvn project or not. I had a situation where I had to do some unusual stuff like copy generated source files from target directory back to the src directory for further processing. In my situation, this was the only option that worked without fuss. Sample code snippet from pom.xml:

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.8</version>
        <executions>
            <execution>
                <phase>process-resources</phase>
                <configuration>
                <tasks>
                    <copy file="${basedir}/target/myome/minifyJsSrcDir/myome.min.js" todir="${basedir}/src/main/webapp/app/minifyJsSrcDir"/>
                </tasks>
                </configuration>
                <goals>
                <goal>run</goal>
                </goals>
            </execution>
        </executions>
</plugin>

Another way is to bundle these things into an artifact using the assembly plugin. Then you can use the dependency plugin to unpack these files where you want. There are also copy goals in the dependency plugin to copy artifacts.


<build>
    <plugins>
        ...
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>2.3</version>
        </plugin>
    </plugins>
    <resources>
        <resource>
            <directory>src/main/java</directory>
            <includes>
                <include> **/*.properties</include>
            </includes>
        </resource>
    </resources>
    ...
</build>

I was able to piece together a number of different sources for this answer:

...
<repository>
    <id>atlassian</id>
    <name>Atlassian Repo</name>
    <url>https://maven.atlassian.com/content/repositories/atlassian-public</url>
</repository>
...
<dependency>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-upload-plugin</artifactId>
    <version>1.1</version>
</dependency>
...
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-upload-plugin</artifactId>
    <version>1.1</version>
    <configuration>
        <serverId>jira-repo</serverId>
        <resourceSrc>
            ${project.build.directory}/${project.build.finalName}.${project.packaging}
        </resourceSrc>
        <resourceDest>opt/jira/webapps</resourceDest> <!-- note: no leading slash -->
        <url>scp://root@jira</url>
    </configuration>
</plugin>
...

From ~/.m2/settings.xml:

...
<servers>
  <server>
    <id>jira-repo</id>
    <username>myusername</username>
    <password>mypassword</password>
  </server>
</servers>
...

Then run the command: (the -X is for debug)

mvn -X upload:upload


In order to copy a file use:

        <plugin>
            <artifactId>maven-resources-plugin</artifactId>
            <version>3.1.0</version>
            <executions>
                <execution>
                    <id>copy-resource-one</id>
                    <phase>install</phase>
                    <goals>
                        <goal>copy-resources</goal>
                    </goals>

                    <configuration>
                        <outputDirectory>${basedir}/destination-folder</outputDirectory>
                        <resources>
                            <resource>
                                <directory>/source-folder</directory>
                                <includes>
                                    <include>file.jar</include>
                                </includes>
                            </resource>
                        </resources>
                    </configuration>
                </execution>
           </executions>
        </plugin>

In order to copy folder with sub-folders use next configuration:

           <configuration>
              <outputDirectory>${basedir}/target-folder</outputDirectory>
              <resources>          
                <resource>
                  <directory>/source-folder</directory>
                  <filtering>true</filtering>
                </resource>
              </resources>              
            </configuration>  

I've had very good experience with copy-maven-plugin. It has a much more convenient and concise syntax in comparison to maven-resources-plugin.


Well, maven is not supposed to be good in doing fine granular tasks, it is not a scripting language like bash or ant, it is rather declarative - you say - i need a war, or an ear, and you get it. However if you need to customize how the war or ear should look like inside, you have a problem. It is just not procedural like ant, but declarative. This have some pros in the beginning, and could have a lot of cons at the end.

I guess the initial concept was to have fine plugins, that "just work" but the reality is different if you do non-standard stuff.

If you however put enough effort in your poms and few custom plugins, you'll get a much better build environment as with ant for example (depends on you project of course, but it gets more and more true for bigger projects).


The maven dependency plugin saved me a lot of time fondling with ant tasks:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>install-jar</id>
            <phase>install</phase>
            <goals>
                <goal>copy</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>...</groupId>
                        <artifactId>...</artifactId>
                        <version>...</version>
                    </artifactItem>
                </artifactItems>
                <outputDirectory>...</outputDirectory>
                <stripVersion>true</stripVersion>
            </configuration>
        </execution>
    </executions>
</plugin>

The dependency:copy is documentend, and has more useful goals like unpack.


To summarize some of the fine answers above: Maven is designed to build modules and copy the results to a Maven repository. Any copying of modules to a deployment/installer-input directory must be done outside the context of Maven's core functionality, e.g. with the Ant/Maven copy command.


I can only assume that your ${project.server.config} property is something custom defined and is outside of the standard directory layout.

If so, then I'd use the copy task.


A generic way to copy arbitrary files is to utilize Maven Wagon transport abstraction. It can handle various destinations via protocols like file, HTTP, FTP, SCP or WebDAV.

There are a few plugins that provide facilities to copy files through the use of Wagon. Most notable are:

  • Out-of-the-box Maven Deploy Plugin

    There is the deploy-file goal. It it quite inflexible but can get the job done:

    mvn deploy:deploy-file -Dfile=/path/to/your/file.ext -DgroupId=foo 
    -DartifactId=bar -Dversion=1.0 -Durl=<url> -DgeneratePom=false
    

    Significant disadvantage to using Maven Deploy Plugin is that it is designated to work with Maven repositories. It assumes particular structure and metadata. You can see that the file is placed under foo/bar/1.0/file-1.0.ext and checksum files are created. There is no way around this.

  • Wagon Maven Plugin

    Use the upload-single goal:

    mvn org.codehaus.mojo:wagon-maven-plugin:upload-single
    -Dwagon.fromFile=/path/to/your/file.ext -Dwagon.url=<url>
    

    The use of Wagon Maven Plugin for copying is straightforward and seems to be the most versatile.


In the examples above <url> can be of any supported protocol. See the list of existing Wagon Providers. For example

  • copying file locally: file:///copy/to
  • copying file to remote host running SSH: scp://host:22/copy/to


The examples above pass plugin parameters in the command line. Alternatively, plugins can be configured directly in POM. Then the invocation will simply be like mvn deploy:deploy-file@configured-execution-id. Or it can be bound to particular build phase.


Please note that for protocols like SCP to work you will need to define an extension in your POM:

<build>
  [...]
  <extensions>
    <extension>
      <groupId>org.apache.maven.wagon</groupId>
      <artifactId>wagon-ssh</artifactId>
      <version>2.12</version>
    </extension>
  </extensions>


If the destination you are copying to requires authentication, credentials can be provided via Server settings. repositoryId/serverId passed to the plugins must match the server defined in the settings.


For a simple copy-tasks I can recommend copy-rename-maven-plugin. It's straight forward and simple to use:

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>com.coderplus.maven.plugins</groupId>
        <artifactId>copy-rename-maven-plugin</artifactId>
        <version>1.0</version>
        <executions>
          <execution>
            <id>copy-file</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>copy</goal>
            </goals>
            <configuration>
              <sourceFile>src/someDirectory/test.environment.properties</sourceFile>
              <destinationFile>target/someDir/environment.properties</destinationFile>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

If you would like to copy more than one file, replace the <sourceFile>...</destinationFile> part with

<fileSets>
  <fileSet>
    <sourceFile>src/someDirectory/test.environment.properties</sourceFile>
    <destinationFile>target/someDir/environment.properties</destinationFile>
  </fileSet>
  <fileSet>
    <sourceFile>src/someDirectory/test.logback.xml</sourceFile>
    <destinationFile>target/someDir/logback.xml</destinationFile>
  </fileSet>                
</fileSets>

Furthermore you can specify multiple executions in multiple phases if needed, the second goal is "rename", which simply does what it says while the rest of the configuration stays the same. For more usage examples refer to the Usage-Page.

Note: This plugin can only copy files, not directories. (Thanks to @james.garriss for finding this limitation.)


The ant solution above is easiest to configure, but I have had luck using the maven-upload-plugin from Atlassian. I was unable to find good documentation, here is how I use it:

<build>
  <plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-upload-plugin</artifactId>
    <version>1.1</version>
    <configuration>
       <resourceSrc>
             ${project.build.directory}/${project.build.finalName}.${project.packaging}
       </resourceSrc>
       <resourceDest>${jboss.deployDir}</resourceDest>
       <serverId>${jboss.host}</serverId>
       <url>${jboss.deployUrl}</url>
     </configuration>
  </plugin>
</build>

The variables like "${jboss.host}" referenced above are defined in my ~/.m2/settings.xml and are activated using maven profiles. This solution is not constrained to JBoss, this is just what I named my variables. I have a profile for dev, test, and live. So to upload my ear to a jboss instance in test environment I would execute:

mvn upload:upload -P test

Here is a snipet from settings.xml:

<server>
  <id>localhost</id>
  <username>username</username>
  <password>{Pz+6YRsDJ8dUJD7XE8=} an encrypted password. Supported since maven 2.1</password>
</server>
...
<profiles>
  <profile>
    <id>dev</id>
    <properties>
      <jboss.host>localhost</jboss.host> 
      <jboss.deployDir>/opt/jboss/server/default/deploy/</jboss.deployDir>
      <jboss.deployUrl>scp://root@localhost</jboss.deployUrl>
    </properties>
  </profile>
  <profile>
    <id>test</id>
    <properties>
       <jboss.host>testserver</jboss.host>
       ...

Notes: The Atlassian maven repo that has this plugin is here: https://maven.atlassian.com/public/

I recommend downloading the sources and looking at the documentation inside to see all the features the plugin provides.

`


Examples related to maven

Maven dependencies are failing with a 501 error Why am I getting Unknown error in line 1 of pom.xml? Why am I getting "Received fatal alert: protocol_version" or "peer not authenticated" from Maven Central? How to resolve Unable to load authentication plugin 'caching_sha2_password' issue Unable to compile simple Java 10 / Java 11 project with Maven ERROR Source option 1.5 is no longer supported. Use 1.6 or later 'react-scripts' is not recognized as an internal or external command How to create a Java / Maven project that works in Visual Studio Code? "The POM for ... is missing, no dependency information available" even though it exists in Maven Repository Java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/exc/InvalidDefinitionException

Examples related to maven-2

Maven:Non-resolvable parent POM and 'parent.relativePath' points at wrong local POM Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project. Plugin org.apache.maven.plugins:maven-clean-plugin:2.5 or one of its dependencies could not be resolved Maven Error: Could not find or load main class MAVEN_HOME, MVN_HOME or M2_HOME Where is my m2 folder on Mac OS X Mavericks Maven won't run my Project : Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec 'mvn' is not recognized as an internal or external command, operable program or batch file SLF4J: Class path contains multiple SLF4J bindings Create local maven repository

Examples related to build-process

how to get docker-compose to use the latest image from repository How do I generate sourcemaps when using babel and webpack? How do I put all required JAR files in a library folder inside the final JAR file with Maven? How do I print a list of "Build Settings" in Xcode project? Running CMake on Windows Why maven? What are the benefits? How to get a dependency tree for an artifact? Can I compile all .cpp files in src/ to .o's in obj/, then link to binary in ./? How do I get Maven to use the correct repositories? Maven: add a dependency to a jar by relative path

Examples related to release-management

How to select different app.config for several build configurations Best practices for copying files with Maven Create a directly-executable cross-platform GUI app using Python