[nginx] How to configure Docker port mapping to use Nginx as an upstream proxy?

Update II

It's now July 16th, 2015 and things have changed again. I've discovered this automagical container from Jason Wilder: https://github.com/jwilder/nginx-proxy and it solves this problem in about as long as it takes to docker run the container. This is now the solution I'm using to solve this problem.

Update

It's now July of 2015 and things have change drastically with regards to networking Docker containers. There are now many different offerings that solve this problem (in a variety of ways).

You should use this post to gain a basic understanding of the docker --link approach to service discovery, which is about as basic as it gets, works very well, and actually requires less fancy-dancing than most of the other solutions. It is limited in that it's quite difficult to network containers on separate hosts in any given cluster, and containers cannot be restarted once networked, but does offer a quick and relatively easy way to network containers on the same host. It's a good way to get an idea of what the software you'll likely be using to solve this problem is actually doing under the hood.

Additionally, you'll probably want to also check out Docker's nascent network, Hashicorp's consul, Weaveworks weave, Jeff Lindsay's progrium/consul & gliderlabs/registrator, and Google's Kubernetes.

There's also the CoreOS offerings that utilize etcd, fleet, and flannel.

And if you really want to have a party you can spin up a cluster to run Mesosphere, or Deis, or Flynn.

If you're new to networking (like me) then you should get out your reading glasses, pop "Paint The Sky With Stars — The Best of Enya" on the Wi-Hi-Fi, and crack a beer — it's going to be a while before you really understand exactly what it is you're trying to do. Hint: You're trying to implement a Service Discovery Layer in your Cluster Control Plane. It's a very nice way to spend a Saturday night.

It's a lot of fun, but I wish I'd taken the time to educate myself better about networking in general before diving right in. I eventually found a couple posts from the benevolent Digital Ocean Tutorial gods: Introduction to Networking Terminology and Understanding ... Networking. I suggest reading those a few times first before diving in.

Have fun!

Original Post

I can't seem to grasp port mapping for Docker containers. Specifically how to pass requests from Nginx to another container, listening on another port, on the same server.

I've got a Dockerfile for an Nginx container like so:

FROM ubuntu:14.04
MAINTAINER Me <[email protected]>

RUN apt-get update && apt-get install -y htop git nginx

ADD sites-enabled/api.myapp.com /etc/nginx/sites-enabled/api.myapp.com
ADD sites-enabled/app.myapp.com /etc/nginx/sites-enabled/app.myapp.com
ADD nginx.conf /etc/nginx/nginx.conf

RUN echo "daemon off;" >> /etc/nginx/nginx.conf

EXPOSE 80 443

CMD ["service", "nginx", "start"]

And then the api.myapp.com config file looks like so:

upstream api_upstream{

    server 0.0.0.0:3333;

}


server {

    listen 80;
    server_name api.myapp.com;
    return 301 https://api.myapp.com/$request_uri;

}


server {

    listen 443;
    server_name api.mypp.com;

    location / {

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
        proxy_pass http://api_upstream;

    }

}

And then another for app.myapp.com as well.

And then I run:

sudo docker run -p 80:80 -p 443:443 -d --name Nginx myusername/nginx

And it all stands up just fine, but the requests are not getting passed-through to the other containers/ports. And when I ssh into the Nginx container and inspect the logs I see no errors.

Any help?

AJB's "Option B" can be made to work by using the base Ubuntu image and setting up nginx on your own. (It didn't work when I used the Nginx image from Docker Hub.)

Here is the Docker file I used:

FROM ubuntu
RUN apt-get update && apt-get install -y nginx
RUN ln -sf /dev/stdout /var/log/nginx/access.log
RUN ln -sf /dev/stderr /var/log/nginx/error.log
RUN rm -rf /etc/nginx/sites-enabled/default
EXPOSE 80 443
COPY conf/mysite.com /etc/nginx/sites-enabled/mysite.com
CMD ["nginx", "-g", "daemon off;"]

My nginx config (aka: conf/mysite.com):

server {
    listen 80 default;
    server_name mysite.com;

    location / {
        proxy_pass http://website;
    }
}

upstream website {
    server website:3000;
}

And finally, how I start my containers:

$ docker run -dP --name website website
$ docker run -dP --name nginx --link website:website nginx

This got me up and running so my nginx pointed the upstream to the second docker container which exposed port 3000.

Using docker links, you can link the upstream container to the nginx container. An added feature is that docker manages the host file, which means you'll be able to refer to the linked container using a name rather than the potentially random ip.

Just found an article from Anand Mani Sankar wich shows a simple way of using nginx upstream proxy with docker composer.

Basically one must configure the instance linking and ports at the docker-compose file and update upstream at nginx.conf accordingly.

I tried using the popular Jason Wilder reverse proxy that code-magically works for everyone, and learned that it doesn't work for everyone (ie: me). And I'm brand new to NGINX, and didn't like that I didn't understand the technologies I was trying to use.

Wanted to add my 2 cents, because the discussion above around linking containers together is now dated since it is a deprecated feature. So here's an explanation on how to do it using networks. This answer is a full example of setting up nginx as a reverse proxy to a statically paged website using Docker Compose and nginx configuration.

TL;DR;

Add the services that need to talk to each other onto a predefined network. For a step-by-step discussion on Docker networks, I learned some things here: https://technologyconversations.com/2016/04/25/docker-networking-and-dns-the-good-the-bad-and-the-ugly/

Define the Network

First of all, we need a network upon which all your backend services can talk on. I called mine web but it can be whatever you want.

docker network create web

Build the App

We'll just do a simple website app. The website is a simple index.html page being served by an nginx container. The content is a mounted volume to the host under a folder content

DockerFile:

FROM nginx
COPY default.conf /etc/nginx/conf.d/default.conf

default.conf

server {
    listen       80;
    server_name  localhost;

    location / {
        root   /var/www/html;
        index  index.html index.htm;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

docker-compose.yml

version: "2"

networks:
  mynetwork:
    external:
      name: web

services:
  nginx:
    container_name: sample-site
    build: .
    expose:
      - "80"
    volumes:
      - "./content/:/var/www/html/"
    networks:
      default: {}
      mynetwork:
        aliases:
          - sample-site

Note that we no longer need port mapping here. We simple expose port 80. This is handy for avoiding port collisions.

Run the App

Fire this website up with

docker-compose up -d

Some fun checks regarding the dns mappings for your container:

docker exec -it sample-site bash
ping sample-site

This ping should work, inside your container.

Build the Proxy

Nginx Reverse Proxy:

Dockerfile

FROM nginx

RUN rm /etc/nginx/conf.d/*

We reset all the virtual host config, since we're going to customize it.

docker-compose.yml

version: "2"

networks:
  mynetwork:
    external:
      name: web


services:
  nginx:
    container_name: nginx-proxy
    build: .
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./conf.d/:/etc/nginx/conf.d/:ro
      - ./sites/:/var/www/
    networks:
      default: {}
      mynetwork:
        aliases:
          - nginx-proxy

Run the Proxy

Fire up the proxy using our trusty

docker-compose up -d

Assuming no issues, then you have two containers running that can talk to each other using their names. Let's test it.

docker exec -it nginx-proxy bash
ping sample-site
ping nginx-proxy

Set up Virtual Host

Last detail is to set up the virtual hosting file so the proxy can direct traffic based on however you want to set up your matching:

sample-site.conf for our virtual hosting config:

  server {
    listen 80;
    listen [::]:80;

    server_name my.domain.com;

    location / {
      proxy_pass http://sample-site;
    }

  }

Based on how the proxy was set up, you'll need this file stored under your local conf.d folder which we mounted via the volumes declaration in the docker-compose file.

Last but not least, tell nginx to reload it's config.

docker exec nginx-proxy service nginx reload

These sequence of steps is the culmination of hours of pounding head-aches as I struggled with the ever painful 502 Bad Gateway error, and learning nginx for the first time, since most of my experience was with Apache.

This answer is to demonstrate how to kill the 502 Bad Gateway error that results from containers not being able to talk to one another.

I hope this answer saves someone out there hours of pain, since getting containers to talk to each other was really hard to figure out for some reason, despite it being what I expected to be an obvious use-case. But then again, me dumb. And please let me know how I can improve this approach.

@gdbj's answer is a great explanation and the most up to date answer. Here's however a simpler approach.

So if you want to redirect all traffic from nginx listening to 80 to another container exposing 8080, minimum configuration can be as little as:

nginx.conf:

server {
    listen 80;

    location / {
        proxy_pass http://client:8080; # this one here
        proxy_redirect off;
    }

}

docker-compose.yml

version: "2"
services:
  entrypoint:
    image: some-image-with-nginx
    ports:
      - "80:80"
    links:
      - client  # will use this one here

  client:
    image: some-image-with-api
    ports:
      - "8080:8080"

Docker docs