Crafted Docker Reverse Proxy

Problem: Routing

The developer teams of one of my customers adapted Docker. After they understood the principle one container - one process and the usage of Docker-Compose they had realizd how easy it is to run and test different deployment-stacks. One for the Master branch, one for Feature/Bugfix-XY and so on. That’s easy on localhost, the developer knows the port mappings of each stack. But what would happen if you have to access these stacks by using a Domain and the requests have to go through some kind of Proxy, e.g. for SSL termination and access control? In this case it would be necessary to reconfigure this Proxy on every new container/stack deployment. If you use multiple Docker or Swarm Hosts and have frequent deployments, it would become very unhandy.

Solution: Convention over Configuration

After some discussion we came up with the idea of using an URL convention which covers all routing-relevant information. The implementation requires a wildcard subdomain and some sort of reverse proxy which could resolve the encoded routing information.

Convention: https://[CONTAINER]-<HOST>.dev.example.com/<CONTEXT>

Request flow

The external Loadbalancer receives all requests from *.dev.example.com and handles the SSL termination and other things. The requests are handed over to some central nginx instances to determine the target Docket Host. The Docker/Swarm-Proxy must forward the request to the container with corresponding name.

Nginx Config

Nginx is one of the smartest webserver out there. It is capable to resolve virtual hosts by regular expressions. We could write a regex to strip <HOST> from the request and proxy to the corresponding Docker server. Here’s the config gist.

server {
  listen 80;
  server_name ~^(.*-)?([a-z0-9]+)\.dev\.example\.com$;

  location / {
    proxy_pass http://docker-host-$2;
    proxy_set_header Host $host;
  }
}

Docker-Proxy

This is the more interesting part. A reverse proxy which is able to find the proper container (and its port) for the incoming request without manually altering the config for each container start/stop.

It should also give helpful hints to the user when the proxying fails.

Resolving

The set of running containers providing any kind of HTTP-Service forms the single source of truth. The agreed convention defines that a HTTP-Service-Container has to expose the environment variable HTTP_PORT, either by docker run --env HTTP_PORT=... or by the Dockerfile definition.

Docker-Proxy can get this knowledge in real time by listening to the Docker-Event-Stream. It emits notifications on container start, stop and other events. The stream is accessible through docker CLI (docker events) or docker.socket.

With the available information from the HTTP-Request and all running containers the Docker-Proxy is able to hand over the request to target IP by involving the resolver algorithm:

// Stripped sample
function resolver($containerName, $context) {
  if ($containerName !== NULL) {
    if isIP($containerName) return [$containerName,$HTTP_PORT];
    if containerExists($containerName) return [getContainerIP($containerName),$HTTP_PORT];
  } else if($context !== NULL) {
    if containerExists($context) return [getContainerIP($context),$HTTP_PORT];
    // search for all containers beginning with `<CONTEXT>`
    // and choose the one with highest leixcal order
    $target = findContainer($context);
    if ($target !== NULL) return [$target,$HTTP_PORT];
  }
  return false; // render error page
}

You can base the algorithm on database lookups or whatever.

The resolving-algorithm returns the IP-Address and the Port of the target container. It’s now easy to forward the request. The Proxy can also apply some further changes like HTTP-Header injection based on various rules, e.g. env-vars.

Implementation

I’ve implemented the Docker-Proxy in Node.js. It’s event based, streaming and I had some experiences accessing the docker.sock from past projects like docker-etcd-registrator. Wiring all together was not too difficult. The docker-modem module abstracts the communication with the Docker API and the http-proxy handles the request forwarding.

Result

Application stacks are now accessible with almost zero configuration and in real time. The developer teams can launch a stack for ie Feature-XY and give the URL feature-xy-host01.dev.example.com to the testing team.

I didn’t publish the source code because it’s specifically written for my customer. But if you’re interested, feel free to contact me.

There are similar projects covering more generic use cases like traefik and docker-gen.