How can I backup a Docker-container with its data-volumes

Question

I ve been using this Docker-image tutum wordpress to demonstrate a Wordpress website  Recently I found out that the image uses volumes for the MySQL-data   So the problem is this  If I want to backup and restore the container I can try to commit an image  and then later delete the container  and create a new container from the committed image  But if I do that the volume gets deleted and all my data is gone   There must be some simple way to backup my container plus its volume-data but I can t find it anywhere

User · Answer

If you want a complete backup  you will need to perform a few steps    Commit the container to an image Save the image Backup the container s volume by creating a tar file of the volume s mount point in the container  Repeat steps 1-3 for the database container as well    Note that doing just a Docker commit of the container to an image does NOT include volumes attached to the container  ref  Docker commit documentation     The commit operation will not include any data contained in volumes mounted inside the container

User · Answer

If you like entering arcane operators from the command line  you   ll love these manual container backup techniques  Keep in mind  there   s a faster and more efficient way to backup containers that   s just as effective  I ve written instructions here  https   www morpheusdata com blog 2017-03-02-how-to-create-a-docker-backup-with-morpheus  Step 1  Add a Docker Host to Any Cloud As explained in a tutorial on the Morpheus support site  you can add a Docker host to the cloud of your choice in a matter of seconds  Start by choosing Infrastructure on the main Morpheus navigation bar  Select Hosts at the top of the Infrastructure window  and click the     Container Hosts    button at the top right   To back up a Docker host to a cloud via Morpheus  navigate to the Infrastructure screen and open the     Container Hosts    menu    Choose a container host type on the menu  select a group  and then enter data in the five fields  Name  Description  Visibility  Select a Cloud and Enter Tags  optional   Click Next  and then configure the host options by choosing a service plan  Note that the Volume  Memory  and CPU count fields will be visible only if the plan you select has custom options enabled   Here is where you add and size volumes  set memory size and CPU count  and choose a network  You can also configure the OS username and password  the domain name  and the hostname  which by default is the container name you entered previously  Click Next  and then add any Automation Workflows  optional  Finally  review your settings and click Complete to save them   Step 2  Add Docker Registry Integration to Public or Private Clouds Adam Hicks describes in another Morpheus tutorial how simple it is to integrate with a private Docker Registry   No added configuration is required to use Morpheus to provision images with Docker   s public hub using the public Docker API    Select Integrations under the Admin tab of the main navigation bar  and then choose the     New Integration    button on the right side of the screen  In the Integration window that appears  select Docker Repository in the Type drop-down menu  enter a name and add the private registry API endpoint  Supply a username and password for the registry you   re using  and click the Save Changes button   Integrate a Docker Registry with a private cloud via the Morpheus    New Integration    dialog box    To provision the integration you just created  choose Docker under Type in the Create Instance dialog  select the registry in the Docker Registry drop-down menu under the Configure tab  and then continue provisioning as you would any Docker container   Step 3  Manage Backups Once you   ve added the Docker host and integrated the registry  a backup will be configured and performed automatically for each instance you provision  Morpheus support provides instructions for viewing backups  creating an instance backup  and creating a server backup

User · Answer

I have created a tool to orchestrate and launch backup of data and mysql containers  simply called docker-backup  There is even a ready-to-use image on the docker hub   It s mainly written in Bash as it is mainly orchestration  It uses duplicity for the actual backup engine  You can currently backup to FTP S  and Amazon S3   The configuration is quite simple  write a config file in YAML describing what to backup and where  and here you go   For data containers  it automatically mount the volumes shared by your container to backup and process it  For mysql containers  it links them and execute a mysqldump bundled with your container and process the result   I wrote it because I use Docker-Cloud which is not up-to-date with recent docker-engine releases and because I wanted to embrace the Docker way by not including any process of backup inside my application containers

User · Answer

The following command will run tar in a container with all named data volumes mounted  and redirect the output into a file   docker run --rm  docker volume list -q   egrep -v     64      awk   print  -v    1    mnt    1    alpine tar -C  mnt -cj    gt  data-volumes tar bz2   Make sure to test the resulting archive in case something went wrong   tar -tjf data-volumes tar bz2

User · Answer

This is a volume-folder-backup way  If you have docker registry infra  This method is very helpful  This uses docker registry for moving the zip file easily    volume folder backup script    bin bash   common bash variables  set these variable before running scripts REPO harbor otcysk org 20443 levee VFOLDER  data mariadb TAG mariadb1   zip local folder for volume files tar cvfz volume-backup tar gz  VFOLDER   copy the zip file to volume-backup container   zip file must be in current folder  docker run -d -v   pwd   temp --name volume-backup ubuntu          bash -c  cd    amp  amp  cp  temp volume-backup tar gz       commit for pushing into REPO docker commit volume-backup  REPO volume-backup  TAG   check gz files in this container  docker run --rm -it --entrypoint bash --name check-volume-backup            REPO volume-backup  TAG   push into REPO docker push  REPO volume-backup  TAG   In another server   pull the image in another server docker pull  REPO volume-backup  TAG   restore files in another server filesystem docker run --rm -v  VFOLDER  VFOLDER --name volume-backup  REPO volume-backup  TAG          bash -c  cd    amp  amp  tar xvfz volume-backup tar gz    Run your image which uses this volume folder  You can make a image which has both one run-image and one volume zip file easily  But I do not recommened for various reasons image size  entry command

User · Answer

I know this is old  but I realize that there isnt a well documented solution to pushing a data container  as backup  to docker hub  I just published a short example on how doing so at https   dzone com articles docker-backup-your-data-volumes-to-docker-hub  Following is the bottom line  The docker tutorial suggest you can backup and restore the data volume locally  We are going to use this technique  add a few more lines to get this backup pushed into docker hub for easy future restoration to any location we desire  So  lets get started  These are the steps to follow   Backup the data volume from the data container named data-container-to-backup  docker run --rm --volumes-from data-container-backup --name tmp-backup -v   pwd   backup ubuntu tar cvf  backup backup tar  folderToBackup   Expand this tar file into a new container so we can commit it as part of its image  docker run -d -v   pwd   backup --name data-backup ubuntu  bin sh -c  cd    amp  amp  tar xvf  backup backup tar    Commit and push the image with a desired tag   VERSION   docker commit data-backup repo data-backup  VERSION docker push repo data-backup  VERSION   Finally  lets clean up  docker rm data-backup docker rmi   docker images -f  dangling true  -q    Now we have an image named data-backup in our repo that is simply a filesystem with the backup files and folders  In order use this image  aka restore from backup   we do the following   Run the data container with the data-backup image  run -v  folderToBackup --entrypoint  bin sh  --name data-container repo data-backup   VERSION    Run your whatEver image with volumes from the data-conainter  docker run --volumes-from data-container repo whatEver   Thats it   I was surprised there is no documentation for this work around  I hope someone find this helpful  I know it took me a while to think about this

User · Answer

If you only need to backup mounted volumes you can just copy folders from your Dockerhost       Note  If you are on Ubuntu  Dockerhost is your local machine  If you are on Mac  Dockerhost is your virtual machine    On Ubuntu  You can find all folders with volumes here   var lib docker volumes  so you can copy them and archive wherever you want   On MAC  It s not so easy as on Ubuntu  You need to copy files from VM   Here is a script of how to copy all folders with volumes from virtual machine  where Docker server is running  to your local machine  We assume that your docker-machine VM named default   docker-machine ssh default sudo cp -v -R  var lib docker volumes   home docker volumes  docker-machine ssh default sudo chmod -R 777  home docker volumes  docker-machine scp -R default  home docker volumes   backup volumes  docker-machine ssh default sudo rm -r  home docker volumes   It is going to create a folder   backup volumes in your current directory and copy all volumes to this folder   Here is a script of how to copy all saved volumes from your local directory    backup volumes  to Dockerhost machine  docker-machine scp -r   backup volumes default  home docker  docker-machine ssh default sudo mv -f  home docker backup volumes  home docker volumes  docker-machine ssh default sudo chmod -R 777  home docker volumes  docker-machine ssh default sudo cp -v -R  home docker volumes  var lib docker   docker-machine ssh default sudo rm -r  home docker volumes   Now you can check if it works by   docker volume ls

User · Answer

If your project uses docker-compose  here is an approach for backing up and restoring your volumes     docker-compose yml  Basically you add db-backup and db-restore services to your docker-compose yml file  and adapt it for the name of your volume   My volume is named dbdata in this example   version   3   services    db      image  percona 5 7     volumes        - dbdata  var lib mysql    db-backup      image  alpine         tty  false     environment        - TARGET dbdata     volumes        -   backup  backup       - dbdata  volume     command  sh -c  tar -cjf  backup    TARGET  tar bz2 -C  volume        db-restore      image  alpine         environment        - SOURCE dbdata     volumes        -   backup  backup       - dbdata  volume     command  sh -c  rm -rf  volume    volume       volume          tar -C  volume  -xjf  backup    SOURCE  tar bz2     Avoid corruption  For data consistency  stop your db container before backing up or restoring  docker-compose stop db   Backing up  To back up to the default destination  backup dbdata tar bz2    docker-compose run --rm db-backup   Or  if you want to specify an alternate target name  do   docker-compose run --rm -e TARGET mybackup db-backup   Restoring  To restore from backup dbdata tar bz2  do   docker-compose run --rm db-restore   Or restore from a specific file using   docker-compose run --rm -e SOURCE mybackup db-restore   I adapted commands from https   loomchild net 2017 03 26 backup-restore-docker-named-volumes  to create this approach

User · Answer

If you have a case as simple as mine was you can do the following    Create a Dockerfile that extends the base image of your container I assume that your volumes are mapped to your filesystem  so you can just add those files folders to your image using ADD folder destination Done    For example  assuming you have the data from the volumes on your home directory  for example at  home mydata you can run the following   DOCKERFILE  home dockerfile bk-myimage docker build --rm --no-cache -t  IMAGENAME  TAG -f  DOCKERFILE  home pirate   Where your DOCKERFILE points to a file like this   FROM user myimage MAINTAINER Danielo Rodr  guez Rivero  lt example gmail com gt   WORKDIR  opt data ADD mydata     The rest of the stuff is inherited from the base image  You can now push that image to docker cloud and your users will have the data available directly on their containers

User · Answer

The problem  You want to backup you image container WITH the data volumes in it but this option is Not out off the box  The straight forward and trivial way would be copy the volumes path and backup the docker image  reload it and and link it both together  but this solution seems to be clumsy and not sustainable and maintainable - You would need to create a cron job that would make this flow each time   Solution  Using dockup - Docker image to backup your Docker container volumes and upload it to s3  Docker   Backup   dockup    dockup will use your AWS credentials to create a new bucket with name as per the environment variable  gets the configured volumes and will be tarballed  gzipped  time-stamped and uploaded to the S3 bucket   Steps     configure the docker-compose yml and attach the env txt configuration file to it  The data should be uploaded to a dedicated secured s3 bucket and ready to be reloaded on DRP executions  in order to verify which volumes path to configure run docker inspect  lt service-name gt  and locate the volumes         Volumes                         etc service-example                           service-example                          Edit the content of the configuration file env txt  and place it on the project path      AWS ACCESS KEY ID  lt key here gt  AWS SECRET ACCESS KEY  lt secret here gt  AWS DEFAULT REGION us-east-1 BACKUP NAME service-backup PATHS TO BACKUP  etc service-example  service-example S3 BUCKET NAME docker-backups example com RESTORE false  Run the dockup container      docker run --rm   --env-file env txt   --volumes-from  lt service-name gt    --name dockup tutum dockup latest     Afterwards verify your s3 bucket contains the relevant data

User · Answer

UPDATE 2  Raw single volume backup bash script      bin bash   This script allows you to backup a single volume from a container   Data in given volume is saved in the current directory in a tar archive  CONTAINER NAME  1 VOLUME NAME  2  usage       echo  Usage   0  container name   volume name     exit 1    if   -z  CONTAINER NAME   then   echo  Error  missing container name parameter     usage fi  if   -z  VOLUME NAME   then   echo  Error  missing volume name parameter     usage fi  sudo docker run --rm --volumes-from  CONTAINER NAME -v   pwd   backup busybox tar cvf  backup backup tar  VOLUME NAME   Raw single volume restore bash script      bin bash   This script allows you to restore a single volume from a container   Data in restored in volume with same backupped path NEW CONTAINER NAME  1  usage       echo  Usage   0  container name     exit 1    if   -z  NEW CONTAINER NAME   then   echo  Error  missing container name parameter     usage fi  sudo docker run --rm --volumes-from  NEW CONTAINER NAME -v   pwd   backup busybox tar xvf  backup backup tar   Usage can be like this     volume backup sh old container  srv www   sudo docker stop old container  amp  amp  sudo docker rm old container   sudo docker run -d --name new container myrepo new container   volume restore sh new container   Assumptions are  backup file is named backup tar  it resides in the same directory as backup and restore script  volume name is the same between containers   UPDATE  It seems to me that backupping volumes from containers is not different from backupping volumes from data containers   Volumes are nothing else than paths linked to a container so the process is the same   I don t know if docker-backup works also for same container volumes but you can use   sudo docker run --rm --volumes-from yourcontainer -v   pwd   backup busybox tar cvf  backup backup tar  data   and   sudo docker run --rm --volumes-from yournewcontainer -v   pwd   backup busybox tar xvf  backup backup tar   END UPDATE  There is this nice tool available which lets you backup and restore docker volumes containers   https   github com discordianfish docker-backup  if you have a container linked to some container volumes like this     docker run --volumes-from my-data-container --name my-server       you can backup all the volumes like this     docker-backup store my-server-backup tar my-server   and restore like this     docker-backup restore my-server-backup tar   Or you can follow the official way   How to port data-only volumes from one host to another

User · Answer

If you just need a simple backup to an archive  you can try my little utility  https   github com loomchild volume-backup  Example  Backup   docker run -v some volume  volume -v  tmp  backup --rm loomchild volume-backup backup archive1   will archive volume named some volume to  tmp archive1 tar bz2 archive file  Restore   docker run -v some volume  volume -v  tmp  backup --rm loomchild volume-backup restore archive1   will wipe and restore volume named some volume from  tmp archive1 tar bz2 archive file   More info  https   medium com  loomchild backup-restore-docker-named-volumes-350397b8e362

User · Answer

Let s say your volume name is data volume  You can use the following commands to backup and restore the volume to and from a docker image named data image   To backup   docker run --rm --mount source data volume destination  data alpine tar -c -f- data   docker run -i --name data container alpine tar -x -f- docker container commit data container data image docker rm data container   To restore   docker run --rm data image tar -c -f- data   docker run -i --rm --mount source data volume destination  data alpine tar -x -f-

User · Answer

if I want to revert the container I can try to commit an image  and then later delete the container  and create a new container from the committed image  But if I do that the volume gets deleted and all my data is gone   As the docker user guide explains  data volumes are meant to persist data outside of a container filesystem  This also ease the sharing of data between multiple containers   While Docker will never delete data in volumes  unless you delete the associated container with docker rm -v   volumes that are not referenced by any docker container are called dangling volumes  Those dangling volumes are difficult to get rid of and difficult to access   This means that as soon as the last container using a volume is deleted  the data volume becomes dangling and its content difficult to acess   In order to prevent those dangling volumes  the trick is to create an additional docker container using the data volume you want to remain   so that there will always be at least that docker container referencing the volume  This way you can delete the docker container running the wordpress app without losing the ease of access to that data volume content    Such containers are called data volume containers      There must be some simple way to back up my container plus volume data but I can t find it anywhere    backup docker images  To backup docker images  use the docker save command that will produce a tar archive that can be used later on to create a new docker image with the docker load command   backup docker containers  You can backup a docker container by different means   by committing a new docker image based on the docker container current state using the docker commit command by exporting the docker container file system as a tar archive using the docker export command  You can later on create a new docker image from that tar archive with the docker import command    Be aware that those commands will only backup the docker container layered file system  This excludes the data volumes   backup docker data volumes  To backup a data volume you can run a new container using the volume you want to backup and executing the tar command to produce an archive of the volume content as described in the docker user guide   In your particular case  the data volume is used to store the data for a MySQL server  So if you want to export a tar archive for this volume  you will need to stop the MySQL server first  To do so you will have to stop the wordpress container   backup the MySQL data  An other way is to remotely connect to the MySQL server to produce a database dump with the mysqldump command  However in order for this to work  your MySQL server must be configured to accept remote connections and also have a user who is allowed to connect remotely  This might not be the case with the wordpress docker image you are using     Edit  Docker recently introduced Docker volume plugins which allow to delegate the handling of volumes to plugins implemented by vendors   The docker run command has a new behavior for the -v option  It is now possible to pass it a volume name  Volumes created in that way are named and easy to reference later on  easing the issues with dangling volumes   Edit 2  Docker introduced the docker volume prune command to delete all dangling volumes easily

[backup] How can I backup a Docker-container with its data-volumes?

Examples related to backup

Examples related to docker

Examples related to linux-containers