I need the latest artifact (for example, a snapshot) from a repository in Artifactory. This artifact needs to be copied to a server (Linux) via a script.
What are my options? Something like Wget / SCP? And how do I get the path of the artifact?
I found some solutions which require Artifactory Pro. But I just have Artifactory, not Artifactory Pro.
Is it possible at all to download from Artifactory without the UI and not having the Pro-Version? What is the experience?
I'm on OpenSUSE 12.1 (x86_64) if that matters.
This question is related to
shell
artifactory
continuous-deployment
I use Nexus and this code works for me—can retrive both release and last snaphsot, depending on repository type:
server="http://example.com/nexus/content/repositories"
repo="snapshots"
name="com.exmple.server"
artifact="com/example/$name"
path=$server/$repo/$artifact
mvnMetadata=$(curl -s "$path/maven-metadata.xml")
echo "Metadata: $mvnMetadata"
jar=""
version=$( echo "$mvnMetadata" | xpath -e "//versioning/release/text()" 2> /dev/null)
if [[ $version = *[!\ ]* ]]; then
jar=$name-$version.jar
else
version=$(echo "$mvnMetadata" | xpath -e "//versioning/versions/version[last()]/text()")
snapshotMetadata=$(curl -s "$path/$version/maven-metadata.xml")
timestamp=$(echo "$snapshotMetadata" | xpath -e "//snapshot/timestamp/text()")
buildNumber=$(echo "$snapshotMetadata" | xpath -e "//snapshot/buildNumber/text()")
snapshotVersion=$(echo "$version" | sed 's/\(-SNAPSHOT\)*$//g')
jar=$name-$snapshotVersion-$timestamp-$buildNumber.jar
fi
jarUrl=$path/$version/$jar
echo $jarUrl
mkdir -p /opt/server/
wget -O /opt/server/server.jar -q -N $jarUrl
You could also use Artifactory Query Language to get the latest artifact.
The following shell script is just an example. It uses 'items.find()' (which is available in the non-Pro version), e.g. items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}}) that searches for files that have a repository name equal to "my-repo" and match all files that start with "my-file". Then it uses the shell JSON parser ./jq to extract the latest file by sorting by the date field 'updated'. Finally it uses wget to download the artifact.
#!/bin/bash
# Artifactory settings
host="127.0.0.1"
username="downloader"
password="my-artifactory-token"
# Use Artifactory Query Language to get the latest scraper script (https://www.jfrog.com/confluence/display/RTF/Artifactory+Query+Language)
resultAsJson=$(curl -u$username:"$password" -X POST http://$host/artifactory/api/search/aql -H "content-type: text/plain" -d 'items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}})')
# Use ./jq to pars JSON
latestFile=$(echo $resultAsJson | jq -r '.results | sort_by(.updated) [-1].name')
# Download the latest scraper script
wget -N -P ./libs/ --user $username --password $password http://$host/artifactory/my-repo/$latestFile
You can use the wget --user=USER --password=PASSWORD .. command, but before you can do that, you must allow artifactory to force authentication, which can be done by unchecking the "Hide Existence of Unauthorized Resources" box at Security/General tab in artifactory admin panel. Otherwise artifactory sends a 404 page and wget can not authenticate to artifactory.
For me the easiest way was to read the last versions of the project with a combination of curl, grep, sort and tail.
My format: service-(version: 1.9.23)-(buildnumber)156.tar.gz
versionToDownload=$(curl -u$user:$password 'https://$artifactory/artifactory/$project/' | grep -o 'service-[^"]*.tar.gz' | sort | tail -1)
Something like the following bash script will retrieve the lastest com.company:artifact snapshot from the snapshot repo:
# Artifactory location
server=http://artifactory.company.com/artifactory
repo=snapshot
# Maven artifact location
name=artifact
artifact=com/company/$name
path=$server/$repo/$artifact
version=$(curl -s $path/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
build=$(curl -s $path/$version/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
jar=$name-$build.jar
url=$path/$version/$jar
# Download
echo $url
wget -q -N $url
It feels a bit dirty, yes, but it gets the job done.
With recent versions of artifactory, you can query this through the api.
If you have a maven artifact with 2 snapshots
name => 'com.acme.derp'
version => 0.1.0
repo name => 'foo'
snapshot 1 => derp-0.1.0-20161121.183847-3.jar
snapshot 2 => derp-0.1.0-20161122.00000-0.jar
Then the full paths would be
and
You would fetch the latest like so:
curl https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-SNAPSHOT.jar
This may be new:
https://artifactory.example.com/artifactory/repo/com/example/foo/1.0.[RELEASE]/foo-1.0.[RELEASE].tgz
For loading module foo from example.com . Keep the [RELEASE] parts verbatim. This is mentioned in the docs but it's not made abundantly clear that you can actually put [RELEASE] into the URL (as opposed to a substitution pattern for the developer).
In case you need to download an artifact in a Dockerfile, instead of using wget or curl or the likes you can simply use the 'ADD' directive:
ADD ${ARTIFACT_URL} /opt/app/app.jar
Of course, the tricky part is determining the ARTIFACT_URL, but there's enough about that in all the other answers.
However, Docker best practises strongly discourage using ADD for this purpose and recommend using wget or curl.
You can use the REST-API's "Item last modified". From the docs, it retuns something like this:
GET /api/storage/libs-release-local/org/acme?lastModified
{
"uri": "http://localhost:8081/artifactory/api/storage/libs-release-local/org/acme/foo/1.0-SNAPSHOT/foo-1.0-SNAPSHOT.pom",
"lastModified": ISO8601
}
Example:
# Figure out the URL of the last item modified in a given folder/repo combination
url=$(curl \
-H 'X-JFrog-Art-Api: XXXXXXXXXXXXXXXXXXXX' \
'http://<artifactory-base-url>/api/storage/<repo>/<folder>?lastModified' | jq -r '.uri')
# Figure out the name of the downloaded file
downloaded_filename=$(echo "${url}" | sed -e 's|[^/]*/||g')
# Download the file
curl -L -O "${url}"
With awk:
curl -sS http://the_repo/com/stackoverflow/the_artifact/maven-metadata.xml | grep latest | awk -F'<latest>' '{print $2}' | awk -F'</latest>' '{print $1}'
With sed:
curl -sS http://the_repo/com/stackoverflow/the_artifact/maven-metadata.xml | grep latest | sed 's:<latest>::' | sed 's:</latest>::'
If you want to download the latest jar between 2 repositores, you can use this solution. I actually use it within my Jenkins pipeline, it works perfectly. Let's say you have a plugins-release-local and plugins-snapshot-local and you want to download the latest jar between these. Your shell script should look like this
NOTE : I use jfrog cli and it's configured with my Artifactory server.
# your repo, you can change it then or pass an argument to the script
# repo = $1 this get the first arg passed to the script
repo=plugins-snapshot-local
# change this by your artifact path, or pass an argument $2
artifact=kshuttle/ifrs16
path=$repo/$artifact
echo $path
~/jfrog rt download --flat $path/maven-metadata.xml version/
version=$(cat version/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
echo "VERSION $version"
~/jfrog rt download --flat $path/$version/maven-metadata.xml build/
build=$(cat build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
echo "BUILD $build"
# change this by your app name, or pass an argument $3
name=ifrs16
jar=$name-$build.jar
url=$path/$version/$jar
# Download
echo $url
~/jfrog rt download --flat $url
def getLatestArtifact(repo, pkg, appName, configDir){
sh """
~/jfrog rt download --flat $repo/$pkg/maven-metadata.xml $configDir/version/
version=\$(cat $configDir/version/maven-metadata.xml | grep latest | sed "s/.*<latest>\\([^<]*\\)<\\/latest>.*/\\1/")
echo "VERSION \$version"
~/jfrog rt download --flat $repo/$pkg/\$version/maven-metadata.xml $configDir/build/
build=\$(cat $configDir/build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\\([^<]*\\)<\\/value>.*/\\1/")
echo "BUILD \$build"
jar=$appName-\$build.jar
url=$repo/$pkg/\$version/\$jar
# Download
echo \$url
~/jfrog rt download --flat \$url
"""
}
def clearDir(dir){
sh """
rm -rf $dir/*
"""
}
node('mynode'){
stage('mysstage'){
def repos = ["plugins-snapshot-local","plugins-release-local"]
for (String repo in repos) {
getLatestArtifact(repo,"kshuttle/ifrs16","ifrs16","myConfigDir/")
}
//optional
clearDir("myConfigDir/")
}
}
This helps alot when you want to get the latest package between 1 or more repos. Hope it helps u too! For more Jenkins scripted pipelines info, visit Jenkins docs.
curl 'http://$artiserver/artifactory/api/storage/$repokey/$path/$version/?lastModified'The above command responds with a JSON with two elements - "uri" and "lastModified"
Fetching the link in the uri returns another JSON which has the "downloadUri" of the artifact.
Fetch the link in the "downloadUri" and you have the latest artefact.
(Requires Pro) to resolve and download latest artifact, if Jenkins Artifactory plugin was used to publish to artifactory in another job:
${repokey}:**/${component}*.jar;status=${STATUS}@${PUBLISH_BUILDJOB}#LATEST=>${targetDir}The role of Artifactory is to provide files for Maven (as well as other build tools such as Ivy, Gradle or sbt). You can just use Maven together with the maven-dependency-plugin to copy the artifacts out. Here's a pom outline to start you off...
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>A group id</groupId>
<artifactId>An artifact id</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<id>copy</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>The group id of your artifact</groupId>
<artifactId>The artifact id</artifactId>
<version>The snapshot version</version>
<type>Whatever the type is, for example, JAR</type>
<outputDirectory>Where you want the file to go</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Just run mvn install to do the copy.
Source: Stackoverflow.com