[docker] how to get docker-compose to use the latest image from repository

I don't know what I'm doing wrong, but I simply cannot get docker-compose up to use the latest image from our registry without first removing the old containers from the system completely. It looks like compose is using the previously started image even though docker-compose pull has fetched a newer image.

I looked at How to get docker-compose to always re-create containers from fresh images? which seemed to be similar to my issue, but none of the provided solutions there work for me, since I'm looking for a solution I can use on the production server and there I don't want to be removing all containers before starting them again (possible data loss?). I would like for compose only to detect the new version of the changed images, pull them and then restart the services with those new images.

I created a simple test project for this in which the only goal is to get a version nr to increase on each new build. The version nr is displayed if I browse to the nginx server that is created (this works as expected locally).

docker version: 1.11.2 docker-compose version: 1.7.1 OS: tested on both CentOS 7 and OS X 10.10 using docker-toolbox

My docker-compose.yml:

version: '2'
services:
  application:
    image: ourprivate.docker.reg:5000/ourcompany/buildchaintest:0.1.8-dev
    volumes:
      - /var/www/html
    tty: true

  nginx:
    build: nginx
    ports:
      - "80:80"
    volumes_from:
      - application
    volumes:
      - ./logs/nginx/:/var/log/nginx
  php:
    container_name: buildchaintest_php_1
    build: php-fpm
    expose:
      - "9000"
    volumes_from:
      - application
    volumes:
      - ./logs/php-fpm/:/var/www/logs

on our jenkins server I run the following to build and tag the image

cd $WORKSPACE && PROJECT_VERSION=$(cat VERSION)-dev
/usr/local/bin/docker-compose rm -f
/usr/local/bin/docker-compose build
docker tag ourprivate.docker.reg:5000/ourcompany/buildchaintest ourprivate.docker.reg:5000/ourcompany/buildchaintest:$PROJECT_VERSION
docker push ourprivate.docker.reg:5000/ourcompany/buildchaintest

this seems to be doing what it's supposed to be since I get a new version tag in our repository each time the build completes and the version nr has been bumped.

If I now run

docker-compose pull && docker-compose -f docker-compose.yml up -d

in a folder on my computer, where the contents is only the docker-compose.yml and the necessary Dockerfiles to build the nginx and php services, the output I get is not the latest version number as has been tagged in the registry or is shown in the docker-compose.yml (0.1.8), but the version before that, which is 0.1.7. However the output of the pull command would suggest that a new version of the image was fetched:

Pulling application (ourprivate.docker.reg:5000/ourcompany/buildchaintest:latest)...
latest: Pulling from ourcompany/buildchaintest
Digest: sha256:8f7a06203005ff932799fe89e7756cd21719cccb9099b7898af2399414bfe62a
Status: Downloaded newer image for docker.locotech.fi:5000/locotech/buildchaintest:0.1.8-dev

Only if I run

docker-compose stop && docker-compose rm -f

and then run the docker-compose up command do I get the new version to show up on screen as expected.

Is this intended behaviour of docker-compose? i.e. should I always do a docker-compose rm -f before running up again, even on production servers? Or am I doing something against the grain here, which is why it's not working?

The goal is to have our build process build and create tagged versions of the images needed in a docker-compose.yml, push those to our private registry and then for the "release to production-step" to simply copy the docker-compose.yml to the production server and run a docker-compose pull && docker-compose -f docker-compose.yml up -d for the new image to start in production. If anyone has tips on this or can point to a best practices tutorial for this kind of setup that would be much appreciated also.

This question is related to docker build-process docker-compose

The answer is


If the docker compose configuration is in a file, simply run:

docker-compose -f appName.yml down && docker-compose -f appName.yml pull && docker-compose -f appName.yml up -d

I am using following command to get latest images

sudo docker-compose down -rmi all

sudo docker-compose up -d


Since 2020-05-07, the docker-compose spec also defines the "pull_policy" property for a service:

version: '3.7'

services:
  my-service:
    image: someimage/somewhere
    pull_policy: always

The docker-compose spec says:

pull_policy defines the decisions Compose implementations will make when it starts to pull images.

Possible values are (tl;dr, check spec for more details):

  • always: always pull
  • never: don't pull (breaks if the image can not be found)
  • missing: pulls if the image is not cached
  • build: always build or rebuild

in order to make sure, that you are using the latest version for your :latest tag from your registry (e.g. docker hub) you need to also pull the latest tag again. in case it changed, the diff will be downloaded and started when you docker-compose up again.

so this would be the way to go:

docker-compose stop
docker-compose rm -f
docker-compose pull   
docker-compose up -d

i glued this into an image that i run to start docker-compose and make sure images stay up-to-date: https://hub.docker.com/r/stephanlindauer/docker-compose-updater/


Option down resolve this problem

I run my compose file:

docker-compose -f docker/docker-compose.yml up -d

then I delete all with down --rmi all

docker-compose -f docker/docker-compose.yml down --rmi all

Stops containers and removes containers, networks, volumes, and images
created by `up`.

By default, the only things removed are:

- Containers for services defined in the Compose file
- Networks defined in the `networks` section of the Compose file
- The default network, if one is used

Networks and volumes defined as `external` are never removed.

Usage: down [options]

Options:
    --rmi type          Remove images. Type must be one of:
                        'all': Remove all images used by any service.
                        'local': Remove only images that don't have a custom tag
                        set by the `image` field.
    -v, --volumes       Remove named volumes declared in the `volumes` section
                        of the Compose file and anonymous volumes
                        attached to containers.
    --remove-orphans    Remove containers for services not defined in the
                        Compose file

But

https://docs.docker.com/compose/reference/up/ -quiet-pull Pull without printing progress information

docker-compose up --quiet-pull

not work ?


The docker-compose documentation for the 'up' command clearly states that it updates the container should the image be changed since the last 'up' was performed:

If there are existing containers for a service, and the service’s configuration or image was changed after the container’s creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes).

So by using 'stop' followed by 'pull' and then 'up' this should therefore avoid issues of lost volumes for the running containers, except of course, for containers whose images have been updated.

I am currently experimenting with this process and will include my results in this comment shortly.


I've seen this occur in our 7-8 docker production system. Another solution that worked for me in production was to run

docker-compose down
docker-compose up -d

this removes the containers and seems to make 'up' create new ones from the latest image.

This doesn't yet solve my dream of down+up per EACH changed container (serially, less down time), but it works to force 'up' to update the containers.


To get the latest images use docker-compose build --pull

I use below command which is really 3 in 1

docker-compose down && docker-compose build --pull && docker-compose up -d

This command will stop the services, pulls the latest image and then starts the services.


I spent half a day with this problem. The reason was that be sure to check where the volume was recorded.

volumes: - api-data:/src/patterns

But the fact is that in this place was the code that we changed. But when updating the docker, the code did not change.

Therefore, if you are checking someone else's code and for some reason you are not updating, check this.

And so in general this approach works:

docker-compose down

docker-compose build

docker-compose up -d


Examples related to docker

standard_init_linux.go:190: exec user process caused "no such file or directory" - Docker What is the point of WORKDIR on Dockerfile? E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them is required for this operation How do I add a user when I'm using Alpine as a base image? docker: Error response from daemon: Get https://registry-1.docker.io/v2/: Service Unavailable. IN DOCKER , MAC How to fix docker: Got permission denied issue pull access denied repository does not exist or may require docker login Docker error: invalid reference format: repository name must be lowercase Docker: "no matching manifest for windows/amd64 in the manifest list entries" OCI runtime exec failed: exec failed: (...) executable file not found in $PATH": unknown

Examples related to build-process

how to get docker-compose to use the latest image from repository How do I generate sourcemaps when using babel and webpack? How do I put all required JAR files in a library folder inside the final JAR file with Maven? How do I print a list of "Build Settings" in Xcode project? Running CMake on Windows Why maven? What are the benefits? How to get a dependency tree for an artifact? Can I compile all .cpp files in src/ to .o's in obj/, then link to binary in ./? How do I get Maven to use the correct repositories? Maven: add a dependency to a jar by relative path

Examples related to docker-compose

E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them is required for this operation How to upgrade docker-compose to latest version How to fix docker: Got permission denied issue Docker error: invalid reference format: repository name must be lowercase Is it safe to clean docker/overlay2/ Docker: How to delete all local Docker images Docker "ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network" How to run docker-compose up -d at system start up? How to create a DB for MongoDB container on start up? How to use local docker images with Minikube?