How to download the latest artifact from Artifactory repository

Question

I need the latest artifact  for example  a snapshot  from a repository in Artifactory  This artifact needs to be copied to a server  Linux  via a script    What are my options  Something like Wget   SCP  And how do I get the path of the artifact   I found some solutions which require Artifactory Pro  But I just have Artifactory  not Artifactory Pro   Is it possible at all to download from Artifactory without the UI and not having the Pro-Version  What is the experience   I m on OpenSUSE 12 1  x86 64  if that matters

User · Answer

If you want to download the latest jar between 2 repositores, you can use this solution. I actually use it within my Jenkins pipeline, it works perfectly. Let's say you have a plugins-release-local and plugins-snapshot-local and you want to download the latest jar between these. Your shell script should look like this

NOTE : I use jfrog cli and it's configured with my Artifactory server.

Use case : Shell script

# your repo, you can change it then or pass an argument to the script
# repo = $1 this get the first arg passed to the script
repo=plugins-snapshot-local
# change this by your artifact path, or pass an argument $2
artifact=kshuttle/ifrs16
path=$repo/$artifact
echo $path
~/jfrog rt download --flat $path/maven-metadata.xml version/
version=$(cat version/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
echo "VERSION $version"
~/jfrog rt download --flat $path/$version/maven-metadata.xml build/
build=$(cat  build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
echo "BUILD $build"
# change this by your app name, or pass an argument $3
name=ifrs16
jar=$name-$build.jar
url=$path/$version/$jar

# Download
echo $url
~/jfrog rt download --flat $url

Use case : Jenkins Pipeline

def getLatestArtifact(repo, pkg, appName, configDir){
    sh """
        ~/jfrog rt download --flat $repo/$pkg/maven-metadata.xml $configDir/version/
        version=\$(cat $configDir/version/maven-metadata.xml | grep latest | sed "s/.*<latest>\\([^<]*\\)<\\/latest>.*/\\1/")
        echo "VERSION \$version"
        ~/jfrog rt download --flat $repo/$pkg/\$version/maven-metadata.xml $configDir/build/
        build=\$(cat  $configDir/build/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\\([^<]*\\)<\\/value>.*/\\1/")
        echo "BUILD \$build"
        jar=$appName-\$build.jar
        url=$repo/$pkg/\$version/\$jar

        # Download
        echo \$url
        ~/jfrog rt download --flat \$url
    """
}

def clearDir(dir){
    sh """
        rm -rf $dir/*
    """

}

node('mynode'){
    stage('mysstage'){
        def repos =  ["plugins-snapshot-local","plugins-release-local"]

        for (String repo in repos) {
            getLatestArtifact(repo,"kshuttle/ifrs16","ifrs16","myConfigDir/")
        }
        //optional
        clearDir("myConfigDir/")
    }
}

This helps alot when you want to get the latest package between 1 or more repos. Hope it helps u too! For more Jenkins scripted pipelines info, visit Jenkins docs.

User · Answer

For me the easiest way was to read the last versions of the project with a combination of curl  grep  sort and tail   My format  service- version  1 9 23 - buildnumber 156 tar gz  versionToDownload   curl -u user  password  https    artifactory artifactory  project     grep -o  service-      tar gz    sort   tail -1

User · Answer

You could also use Artifactory Query Language to get the latest artifact   The following shell script is just an example  It uses  items find     which is available in the non-Pro version   e g  items find    repo      eq   my-repo     name      match     my-file      that searches for files that have a repository name equal to  my-repo  and match all files that start with  my-file   Then it uses the shell JSON parser   jq to extract the latest file by sorting by the date field  updated   Finally it uses wget to download the artifact      bin bash    Artifactory settings host  127 0 0 1  username  downloader  password  my-artifactory-token     Use Artifactory Query Language to get the latest scraper script  https   www jfrog com confluence display RTF Artifactory Query Language  resultAsJson   curl -u username   password  -X POST  http    host artifactory api search aql -H  content-type  text plain  -d  items find    repo      eq   my-repo     name      match     my-file           Use   jq to pars JSON latestFile   echo  resultAsJson   jq -r   results   sort by  updated   -1  name      Download the latest scraper script wget -N -P   libs  --user  username --password  password http    host artifactory my-repo  latestFile

User · Answer

Artifactory has a good extensive REST-API and almost anything that can be done in the UI  perhaps even more  can also be done using simple HTTP requests   The feature that you mention - retrieving the latest artifact  does indeed require the Pro edition  but it can also be achieved with a bit of work on your side and a few basic scripts   Option 1 - Search    Perform a GAVC search on a set of group ID and artifact ID coordinates to retrieve all existing versions of that set  then you can use any version string comparison algorithm to determine the latest version   Option 2 - the Maven way    Artifactory generates a standard XML metadata that is to be consumed by Maven  because Maven is faced with the same problem - determining the latest version  The metadata lists all available versions of an artifact and is generated for every artifact level folder  with a simple GET request and some XML parsing  you can discover the latest version

User · Answer

The role of Artifactory is to provide files for Maven  as well as other build tools such as Ivy  Gradle or sbt   You can just use Maven together with the maven-dependency-plugin to copy the artifacts out  Here s a pom outline to start you off      lt project xmlns  http   maven apache org POM 4 0 0  xmlns xsi  http   www w3 org 2001 XMLSchema-instance           xsi schemaLocation  http   maven apache org POM 4 0 0 http   maven apache org xsd maven-4 0 0 xsd  gt       lt modelVersion gt 4 0 0 lt  modelVersion gt       lt groupId gt A group id lt  groupId gt       lt artifactId gt An artifact id lt  artifactId gt       lt version gt 1 0 0-SNAPSHOT lt  version gt       lt packaging gt pom lt  packaging gt        lt build gt           lt plugins gt               lt plugin gt                   lt groupId gt org apache maven plugins lt  groupId gt                   lt artifactId gt maven-dependency-plugin lt  artifactId gt                   lt version gt 2 3 lt  version gt                   lt executions gt                       lt execution gt                           lt id gt copy lt  id gt                           lt phase gt package lt  phase gt                           lt goals gt                               lt goal gt copy lt  goal gt                           lt  goals gt                           lt configuration gt                               lt artifactItems gt                                   lt artifactItem gt                                       lt groupId gt The group id of your artifact lt  groupId gt                                       lt artifactId gt The artifact id lt  artifactId gt                                       lt version gt The snapshot version lt  version gt                                       lt type gt Whatever the type is  for example  JAR lt  type gt                                       lt outputDirectory gt Where you want the file to go lt  outputDirectory gt                                   lt  artifactItem gt                               lt  artifactItems gt                           lt  configuration gt                       lt  execution gt                   lt  executions gt               lt  plugin gt           lt  plugins gt       lt  build gt   lt  project gt    Just run mvn install to do the copy

User · Answer

Using shell unix tools   curl  http    artiserver artifactory api storage  repokey  path  version  lastModified    The above command responds with a JSON with two elements -  uri  and  lastModified    Fetching the link in the uri returns another JSON which has the  downloadUri  of the artifact  Fetch the link in the  downloadUri  and you have the latest artefact    Using Jenkins Artifactory plugin   Requires Pro  to resolve and download latest artifact  if Jenkins Artifactory plugin was used to publish to artifactory in another job    Select Generic Artifactory Integration Use Resolved Artifacts as   repokey       component   jar status   STATUS    PUBLISH BUILDJOB  LATEST  gt   targetDir

User · Answer

I use Nexus and this code works for me mdash can retrive both release and last snaphsot  depending on repository type    server  http   example com nexus content repositories  repo  snapshots  name  com exmple server  artifact  com example  name  path  server  repo  artifact mvnMetadata   curl -s   path maven-metadata xml   echo  Metadata   mvnMetadata  jar    version    echo   mvnMetadata    xpath -e    versioning release text    2 gt   dev null  if     version               then   jar  name- version jar else   version   echo   mvnMetadata    xpath -e    versioning versions version last    text       snapshotMetadata   curl -s   path  version maven-metadata xml     timestamp   echo   snapshotMetadata    xpath -e    snapshot timestamp text       buildNumber   echo   snapshotMetadata    xpath -e    snapshot buildNumber text       snapshotVersion   echo   version    sed  s   -SNAPSHOT      g     jar  name- snapshotVersion- timestamp- buildNumber jar fi jarUrl  path  version  jar echo  jarUrl mkdir -p  opt server  wget -O  opt server server jar -q -N  jarUrl

User · Answer

In case you need to download an artifact in a Dockerfile  instead of using wget or curl or the likes you can simply use the  ADD  directive   ADD   ARTIFACT URL   opt app app jar  Of course  the tricky part is determining the ARTIFACT URL  but there s enough about that in all the other answers   However  Docker best practises strongly discourage using ADD for this purpose and recommend using wget or curl

User · Answer

Something like the following bash script will retrieve the lastest com company artifact snapshot from the snapshot repo     Artifactory location server http   artifactory company com artifactory repo snapshot    Maven artifact location name artifact artifact com company  name path  server  repo  artifact version   curl -s  path maven-metadata xml   grep latest   sed  s    lt latest gt      lt      lt   latest gt     1    build   curl -s  path  version maven-metadata xml   grep   lt value gt     head -1   sed  s    lt value gt      lt      lt   value gt     1    jar  name- build jar url  path  version  jar    Download echo  url wget -q -N  url   It feels a bit dirty  yes  but it gets the job done

User · Answer

This may be new   https   artifactory example com artifactory repo com example foo 1 0  RELEASE  foo-1 0  RELEASE  tgz  For loading module foo from example com    Keep the  RELEASE  parts verbatim   This is mentioned in the docs but it s not made abundantly clear that you can actually put  RELEASE  into the URL  as opposed to a substitution pattern for the developer

User · Answer

You can use the REST-API s  Item last modified   From the docs  it retuns something like this   GET  api storage libs-release-local org acme lastModified    uri    http   localhost 8081 artifactory api storage libs-release-local org acme foo 1 0-SNAPSHOT foo-1 0-SNAPSHOT pom    lastModified   ISO8601     Example     Figure out the URL of the last item modified in a given folder repo combination url   curl       -H  X-JFrog-Art-Api  XXXXXXXXXXXXXXXXXXXX         http    lt artifactory-base-url gt  api storage  lt repo gt   lt folder gt  lastModified     jq -r   uri     Figure out the name of the downloaded file downloaded filename   echo    url     sed -e  s         g     Download the file curl -L -O    url

User · Answer

With awk        curl  -sS http   the repo com stackoverflow the artifact maven-metadata xml   grep latest   awk -F  lt latest gt     print  2     awk -F  lt  latest gt     print  1     With sed       curl  -sS http   the repo com stackoverflow the artifact maven-metadata xml   grep latest   sed  s  lt latest gt       sed  s  lt  latest gt

User · Answer

You can use the wget --user USER --password PASSWORD    command  but before you can do that  you must allow artifactory to force authentication  which can be done by unchecking the  Hide Existence of Unauthorized Resources  box at Security General tab in artifactory admin panel  Otherwise artifactory sends a 404 page and wget can not authenticate to artifactory

User · Answer

With recent versions of artifactory  you can query this through the api    https   www jfrog com confluence display RTF Artifactory REST API ArtifactoryRESTAPI-RetrieveLatestArtifact  If you have a maven artifact with 2 snapshots  name     com acme derp  version    0 1 0 repo name     foo  snapshot 1    derp-0 1 0-20161121 183847-3 jar snapshot 2    derp-0 1 0-20161122 00000-0 jar    Then the full paths would be     https   artifactory example com artifactory foo com acme derp 0 1 0-SNAPSHOT derp-0 1 0-20161121 183847-3 jar   and     https   artifactory example com artifactory foo com acme derp 0 1 0-SNAPSHOT derp-0 1 0-20161122 00000-0 jar   You would fetch the latest like so   curl https   artifactory example com artifactory foo com acme derp 0 1 0-SNAPSHOT derp-0 1 0-SNAPSHOT jar

[shell] How to download the latest artifact from Artifactory repository?

Use case : Shell script

Use case : Jenkins Pipeline

Examples related to shell

Examples related to artifactory

Examples related to continuous-deployment