Communication between multiple docker-compose projects

Question

I have two separate docker-compose yml files in two different folders      front docker-compose yml   api docker-compose yml   How can I make sure that a container in front can send requests to a container in api   I know that --default-gateway option can be set using docker run for an individual container  so that a specific IP address can be assigned to this container  but it seems that this option is not available when using docker-compose   Currently I end up doing a docker inspect my api container id and look at the gateway in the output  It works but the problem is that this IP is randomly attributed  so I can t rely on it   Another form of this question might thus be    Can I attribute a fixed IP address to a particular container using docker-compose    But in the end what I m looking after is    How can two different docker-compose projects communicate with each other

User · Accepted Answer

You just need to make sure that the containers you want to talk to each other are on the same network. Networks are a first-class docker construct, and not specific to compose.

# front/docker-compose.yml
version: '2'
services:
  front:
    ...
    networks:
      - some-net
networks:
  some-net:
    driver: bridge

...

# api/docker-compose.yml
version: '2'
services:
  api:
    ...
    networks:
      - front_some-net
networks:
  front_some-net:
    external: true

Note: Your app’s network is given a name based on the “project name”, which is based on the name of the directory it lives in, in this case a prefix front_ was added

They can then talk to each other using the service name. From front you can do ping api and vice versa.

User · Answer

The previous posts information is correct  but it does not have details on how to link containers  which should be connected as  external links    Hope this example make more clear to you    Suppose you have app1 docker-compose yml  with two services  svc11 and svc12   and app2 docker-compose yml with two more services  svc21 and svc22  and suppose you need to connect in a crossed fashion  svc11 needs to connect to svc22 s container svc21 needs to connect to svc11 s container    So the configuration should be like this   this is app1 docker-compose yml    version   2  services      svc11          container name  container11                      networks              - default   this network             - app2 default   external network         external links              - container22 container22                  svc12         container name  container12              networks      default    this network  app1          driver  bridge     app2 default    external network  app2          external  true   this is app2 docker-compose yml    version   2  services      svc21          container name  container21                      networks              - default   this network  app2              - app1 default   external network  app1          external links              - container11 container11                  svc22         container name  container22              networks      default    this network  app2          driver  bridge     app1 default    external network  app1          external  true

User · Answer

UPDATE   As of compose file version 3 5  This now works  version   quot 3 5 quot  services    proxy      image  hello-world     ports        -  quot 80 80 quot      networks        - proxynet  networks    proxynet      name  custom network  docker-compose up -d will join a network called  custom network    If it doesn t exist  it will be created  root ubuntu-s-1vcpu-1gb-tor1-01    docker-compose up -d Creating network  quot custom network quot  with the default driver Creating root proxy 1     done  Now  you can do this  version   quot 2 quot  services    web      image  hello-world     networks        - my-proxy-net networks    my-proxy-net      external        name  custom network  This will create a container that will be on the external network  I can t find any reference in the docs yet but it works

User · Answer

Another option is just running up the first module with the  docker-compose  check the ip related with the module  and connect the second module with the previous net like external  and pointing the internal ip  example app1 - new-network created in the service lines  mark as external  true at the bottom app2 - indicate the  new-network  created by app1 when goes up   mark as external  true at the bottom  and set in the config to connect  the ip that app1 have in this net   With this  you should be able to talk with each other   this way is just for local-test focus  in order to don t do an over complex configuration    I know is very  patch way  but works for me and I think is so simple some other can take advantage of this

User · Answer

All containers from api can join the front default network with following config     api docker-compose yml       networks    default      external        name  front default   See docker compose guide  using a pre existing network  see at the bottom

User · Answer

Just a small adittion to  johnharris85 s great answer  when you are running a docker compose file  a  default  network is created so you can just add it to the other compose file as an external network     front docker-compose yml  version   2     services         front service                   api docker-compose yml version   2  services    api service              networks        - front default networks    front default      external  true   For me this approach was more suited because I did not own the first docker-compose file and wanted to communicate with it

User · Answer

version   2  services    bot      build        volumes        -     home node        -  home node node modules     networks        - my-rede     mem limit  100m     memswap limit  100m     cpu quota  25000     container name  236948199393329152 585042339404185600 bot     command  node index js     environment        NODE ENV  production networks    my-rede      external        name  name rede externa

User · Answer

If you are    trying to communicate between two containers from different docker-compose projects and don t want to use the same network  because let s say they would have PostgreSQL or Redis container on the same port and you would prefer to not changing these ports and not use it at the same network   developing locally and want to imitate communication between two docker compose projects running two docker-compose projects on localhost developing especially Django apps or Django Rest Framework  drf  API and running app inside container on some exposed port getting Connection refused while trying to communicate between two containers   And you want to    container api a communicate to api b  or vice versa  without the same  docker network      example below   you can use  host  of the second container as IP of your computer and port that is mapped from inside Docker container  You can obtain IP of your computer with this script  from  Finding local IP addresses using Python  39 s stdlib    import socket def get ip        s   socket socket socket AF INET  socket SOCK DGRAM      try            doesn t even have to be reachable         s connect   10 255 255 255   1           IP   s getsockname   0      except          IP    127 0 0 1      finally          s close       return IP   Example   project api a docker-compose yml   networks    app-tier      driver  bridge  services    api      container name  api a     image  api a latest     depends on        - postgresql     networks        - app-tier   inside api a container you are running Django app  manage py runserver 0 0 0 0 8000  and second docker-compose yml from other project   project api b docker-compose-yml     networks    app-tier      driver  bridge  services    api      container name  api b     image  api b latest     depends on        - postgresql     networks        - app-tier   inside api b container you are running Django app  manage py runserver 0 0 0 0 8001  And trying to connect from container api a to api b then URL of api b container will be  http    lt get ip from script above gt  8001   It can be especially valuable if you are using even more than two three or more  docker-compose projects and it s hard to provide common network for all of it - it s good workaround and solution

User · Answer

UPDATE  As of docker-compose file version 3 5  I came across a similar problem and I solved it by adding a small change in one of my docker-compose yml project  For instance  we have two API s scoring and ner  Scoring API needs to send a request to the ner API for processing the input request  In order to do that they both are supposed to share the same network  Note  Every container has its own network which is automatically created at the time of running the app inside docker  For example ner API network will be created like ner default and scoring API network will be named as scoring default  This solution will work for version   3   As in the above scenario  my scoring API wants to communicate with ner API then I will add the following lines  This means Whenever I create the container for ner API then it automatically added to the scoring default network  networks    default        external          name  scoring default  ner docker-compose yml version   3  services    ner      container name   quot ner api quot      build             networks    default        external          name  scoring default  scoring docker-compose yml version   3  services    api      build             We can see this how the above containers are now a part of the same network called  scoring default using the command   docker inspect scoring default         quot Name quot    quot scoring default quot                     quot Containers quot          quot 14a6   28bf quot              quot Name quot    quot ner api quot            quot EndpointID quot    quot 83b7   d6291 quot            quot MacAddress quot    quot 0     quot            quot IPv4Address quot    quot 0 0     quot            quot IPv6Address quot    quot  quot              quot 7b32   90d1 quot              quot Name quot    quot scoring api quot            quot EndpointID quot    quot 311   280d quot            quot MacAddress quot    quot 0     3 quot            quot IPv4Address quot    quot 1   0 quot            quot IPv6Address quot    quot  quot

User · Answer

For using another docker-compose network you just do these to share networks between docker-compose           Run the first docker-compose project by up -d   Find the network name of the first docker-compose by  docker network ls It contains the name of the root directory project    Then use that name by this structure at below in the second docker-compose file       second docker-compose yml  version   3  services    service-on-second-compose     Define any names that you want                        networks        -  lt put it here the network name that comes from  docker network ls   gt   networks    -  lt put it here the network name that comes from  docker network ls   gt       external  true

User · Answer

I would ensure all containers are docker-compose d to the same network by composing them together at the same time  using   docker compose --file   front docker-compose yml --file   api docker-compose yml up -d

User · Answer

Since Compose 1 18  spec 3 5   you can just override the default network using your own custom name for all Compose YAML files you need  It is as simple as appending the following to them   networks    default      name  my-app      The above assumes you have version set to 3 5  or above if they don t deprecate it in 4      Other answers have pointed the same  this is a simplified summary

User · Answer

You can add a  env file in all your projects containing COMPOSE PROJECT NAME somename   COMPOSE PROJECT NAME overrides the prefix used to name resources  as such all your projects will use somename default as their network  making it possible for services to communicate with each other as they were in the same project   NB  You ll get warnings for  orphaned  containers created from other projects

[networking] Communication between multiple docker-compose projects

Examples related to networking

Examples related to docker

Examples related to docker-compose