Static traefik config for use as docker reverse proxy (avoiding exposure of docker socket)

Hi there,

i am currently migrating my old raspi server and want to use containerization on my Pi4 replacement.
It was quite a learning curve to rebuild the services as docker containers, but I am quite satisfied with the outcome so far.

The next piece of the puzzle for me would be, to utilize lets encrypt certificates and a reverse proxy for my setup.
That was when I learned about traefik and it's abilities.

But where there is light, there is also shadow... As I was researching configuration examples I came across this POST mentioning that exposing the docker socket to a container is risky from security standpoint.
The last comment in the discussion mentioned that with traefik_v2 the issue could be avoided with the file provider, thus creating a static configuration for traefik.

I am not planning on relying on the dynamic configuration, even though it would be nice to have. A static configuration would be ok with me, as I do not intend to alter the services/configuration much and I would value the increased security more.

My question is, what would a secure, reliable traefik reverse proxy configuration for docker containers look like? All examples I found during research were either using older traefik versions or exposing the docker socket to traefik...

My main points which I would like to achieve are:

  • Already solved: Acquiring let's encrypt certs for the dedyn domain I am using via dns01 challenge.
  • I would like to have a central entrance to multiple services behind the traefik proxy (ideally only making the central entrance point known to the internet like gatekeeper.mydomain.example.com) but also have the possibility to call a service directly like nc.gatekeeper.mydomain.example.com or gatekeeper.mydomain.example.com/nc would that be possible?
  • http redirection, e.g. when called via a http entrypoint it should be redirected to the https version with a valid cert
  • a solution that also works with docker-swarm. Currently I leverage the passing of external secrets to the docker containers via the docker-swarm agent and would like to keep it this way.

I came up with a little test configuration after combining some tutorials, but it is not a satisfying solution yet.

For an example, lets say I have three docker containers. A nextcloud image, a database and a adminer instance and of course the traefik container.
I want traefik to be the reverse proxy for these containers, only exposing traefik to the big bad internet and keeping the applications safe. How would I be doing this without involving the docker socket?

Test configuration so far:

docker-compose.yaml


services:

  traefik:
    image: traefik:v2.0.4
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - TZ=Europe/Berlin
      - EXEC_PATH=/etc/traefik/domain_dns
      - DOMAIN_TOKEN=d42d9cd98f00b204e9345998ecf8427e
      - DOMAIN_NAME=mydomain.example.com
    volumes:
      - ./traefik.yml:/etc/traefik/traefik.yml
      - ./dynamic_conf.yml:/etc/traefik/dynamic_conf.yml
      - ./acme.json:/acme.json
      - ./domain_dns:/etc/traefik/domain_dns


  nextcloud:
    image: nextcloud:stable-apache
    volumes:
      - "./nc/data:/var/www/html/data"
      - "./nc/custom_apps:/var/www/html/custom_apps"
      - "./nc/config:/var/www/html/config"
    environment:
      - MYSQL_HOST=db
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=somepassword
    ports:
      - 127.0.0.1:8084:80


  db:
    image: linuxserver/mariadb:arm32v7-110.4.10mariabionic-ls42
    restart: always
    volumes:
      - "./db:/config"
    env_file:
      - "db.env"


  adminer:
    image: adminer:4.7.4-standalone
    restart: always
    ports:
      - 127.0.0.1:8085:8080

traefik.yml

  level: DEBUG

serversTransport:
  insecureSkipVerify: true

entryPoints:
  web:
    address: ":80"

  web-secure:
    address: ":443"

api:
  insecure: true
  dashboard: true

providers:
  file:
    filename: "/etc/traefik/dynamic_conf.yml"
    watch: true

certificatesResolvers:
  sample:
    acme:
      email: admin@mydomain.example.com
      storage: acme.json
      dnsChallenge:
        provider: exec
        delayBeforeCheck: 0

dynamic_conf.yml

http:
  routers:
    router0:
      entyPoints:
      - web
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && Path(`/nc`)"
      middlewares:
      - redirect
    router1:
      entyPoints:
      - web-secure
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && Path(`/nc`)"
      middlewares:
      - ncHeader
      - nc-replacepath
      tls:
        certResolver: sample
    router2:
      entyPoints:
      - web
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`) && Path(`/ad`)"
      middlewares:
      - redirect
    router3:
      entyPoints:
      - web-secure
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`) && Path(`/ad`)"
      middlewares:
      - adHeader
      - ad-replacepath
      tls:
        certResolver: sample

  services:
    nextcloud:
      loadBalancer:
        #prevents the triggering of the "safe domains" feature of apache
        passHostHeader: false
        servers:
          - url: "http://127.0.0.1:8084/"
    adminer:
      loadBalancer:
        servers:
          - url: "http://127.0.0.1:8085/"

  middlewares:
    nc-replacepath:
      replacePath:
        path: ""
    ncHeader:
      headers:
        customRequestHeaders:
          X-Replaced-Path: "/nc"
    ad-replacepath:
      replacePath:
        path: ""
    adHeader:
      headers:
        customRequestHeaders:
          X-Replaced-Path: "/ad"
    redirect:
      redirectScheme:
        scheme: https

Regards Stephan

In which ways does it not satisfy you?

Yeah sorry I forgot to mention the issues, I was a bit in a hurry...

So far if I bind the docker containers to localhost, I get an 502 Bad Gateway when I try to access them by calling raspberrypi.mydomain.example.com/nc but I think this is because traefik then tries to call his Loopback inside the container and fails... What would be the right syntax/configuration for redirecting to the host loopback?

If I instead use the host ip address it works with 50% of the containers (e.g. I can access the adminer webinterface), but the apache server in the nextcloud container has "trusted domains" configured and presents me with an error page "acces over a non trusted domain". I already added the raspberrypi.mydomain.example.com:443 to the config.php of the webserver but so far it did not solve the problem...

regards Stephan

Ok, I tinkered a bit with the configuration today. This is what it looks like now:

docker-compose.yaml

services:

  traefik:
    image: traefik:v2.0.4
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - TZ=Europe/Berlin
      - EXEC_PATH=/etc/traefik/domain_dns
      - DOMAIN_TOKEN=d42d9cd98f00b204e9345998ecf8427e
      - DOMAIN_NAME=mydomain.example.com
    volumes:
      - ./traefik.yml:/etc/traefik/traefik.yml
      - ./dynamic_conf.yml:/etc/traefik/dynamic_conf.yml
      - ./acme.json:/acme.json
      - ./domain_dns:/etc/traefik/domain_dns


  nextcloud:
    image: nextcloud:stable-apache
    volumes:
      - "./nc/data:/var/www/html/data"
      - "./nc/custom_apps:/var/www/html/custom_apps"
      - "./nc/config:/var/www/html/config"
    environment:
      - MYSQL_HOST=db
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=somepassword
    ports:
      - "8084:80"


  db:
    image: linuxserver/mariadb:arm32v7-110.4.10mariabionic-ls42
    restart: always
    volumes:
      - "./db:/config"
    env_file:
      - "db.env"


  adminer:
    image: adminer:4.7.4-standalone
    restart: always
    ports:
      - "8085:8080"

traefik.yml

 level: DEBUG

serversTransport:
  insecureSkipVerify: true

entryPoints:
  web:
    address: ":80"

  web-secure:
    address: ":443"

api:
  insecure: true
  dashboard: true

providers:
  file:
    filename: "/etc/traefik/dynamic_conf.yml"
    watch: true

certificatesResolvers:
  sample:
    acme:
      email: admin@mydomain.example.com
      storage: acme.json
      dnsChallenge:
        provider: exec
        delayBeforeCheck: 0

dynamic_conf.yml

http:
  routers:
    router0:
      entyPoints:
      - web
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && PathPrefix(`/nc`)"
      middlewares:
      - redirect
    router1:
      entyPoints:
      - web-secure
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && PathPrefix(`/nc`)"
      middlewares:
      - removeServiceSelector
      - ncHeader
      tls:
        certResolver: sample
    router2:
      entyPoints:
      - web
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`)  && PathPrefix(`/ad`)"
      middlewares:
      - redirect
    router3:
      entyPoints:
      - web-secure
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`)  && PathPrefix(`/ad`)"
      middlewares:
      - removeServiceSelector
      tls:
        certResolver: sample

  services:
    nextcloud:
      loadBalancer:
        servers:
          - url: "http://raspberrypi.mydomain.example.com:8084/"
    adminer:
      loadBalancer:
        servers:
          - url: "http://raspberrypi.mydomain.example.com:8085/"

  middlewares:
    ncHeader:
      headers:
        customResponseHeaders:
          stsPreload: true
          stsSeconds: 15552000
    removeServiceSelector:
      stripPrefix:
        prefixes:
          - "/nc"
          - "/ad"
        forceSlash: false
    redirect:
      redirectScheme:
        scheme: https

With this configuration when called http://raspberrypi.mydomain.example.com/ad I get the adminer webinterface with a valid letsencrypt cert and everything works fine.

But when I try the nextcloud subdomain http://raspberrypi.mydomain.example.com/nc I get an ERR_SSL_PROTOCOL_ERROR. At the same time also saying it is a secure connection with a valid letsencrypt certificate.

I got this far by modifying the apache webserver config of the nextcloud container, now looking like this:

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'asdfasdfadsf',
  'passwordsalt' => 'asdfasdfasdfasdfasdfasdfasdfas',
  'secret' => 'asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf',
  'trusted_domains' =>
  array (
    0 => '192.168.178.20:8084',
    1 => 'raspberrypi.mydomain.example.com',
  ),
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '16.0.5.1',
  'overwrite.cli.url' => 'https://raspberrypi.mydomain.example.com:8084/',
  'dbname' => 'nextcloud',
  'dbhost' => 'db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'asdfasdfasdf',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 0,
  'mysql.utf8mb4' => true,
  'trusted_proxies' => ['traefik'],
  'overwritehost' => 'raspberrypi.mydomain.example.com:8084',
  'overwriteprotocol' => 'https',
);

but now I am stuck...

regards Stephan

It looks like you are trying to access http://raspberrypi.mydomain.example.com:8084/ over http whereas it requires https.

Ok, I got it working. :smiley:

After a bit of tweaking the nextcloud config.php it now runs as it should.

For anyone interested in my working configuration:
I put the traefik container into his own docker-compose file, the remaining services connect to him via the external "web" network (e.g. you need to create it first with "docker network create web").
The database should not be exposed, therefore it resides in a local network (and has no connection to the "web" network).

The STS headers in the dynamic_config.yml are not complete and need a bit of tuning (suggestions anyone?).

docker-compose.yml for traefik container

version: "3.1"

networks:
  web:
    external: true

services:

  traefik:
    image: traefik:v2.0.4
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - TZ=Europe/Berlin
      - EXEC_PATH=/etc/traefik/domain_dns
      - DOMAIN_TOKEN=d42d9cd98f00b204e9345998ecf8427e
      - DOMAIN_NAME=mydomain.example.com
    volumes:
      - ./traefik.yml:/etc/traefik/traefik.yml
      - ./dynamic_conf.yml:/etc/traefik/dynamic_conf.yml
      - ./acme.json:/acme.json
      - ./domain_dns:/etc/traefik/domain_dns
    networks:
      - web

docker-compose.yml for service containers

version: "3.1"

networks:
  web:
    external: true
  internal:
    external: false

services:  

  nextcloud:
    image: nextcloud:stable-apache
    volumes:
      - "./nc/data:/var/www/html/data"
      - "./nc/custom_apps:/var/www/html/custom_apps"
      - "./nc/config:/var/www/html/config"
    environment:
      - MYSQL_HOST=db
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD=somepassword
    networks:
      - internal
      - web

  db:
    image: linuxserver/mariadb:arm32v7-110.4.10mariabionic-ls42
    restart: always
    volumes:
      - "./db:/config"
    env_file:
      - "db.env"
    networks:
      - internal

  adminer:
    image: adminer:4.7.4-standalone
    restart: always
    networks:
      - internal
      - web

traefik.yml

level: DEBUG

serversTransport:
  insecureSkipVerify: true

entryPoints:
  web:
    address: ":80"

  web-secure:
    address: ":443"

api:
  insecure: true
  dashboard: true

providers:
  file:
    filename: "/etc/traefik/dynamic_conf.yml"
    watch: true

certificatesResolvers:
  sample:
    acme:
      email: admin@mydomain.example.com
      storage: acme.json
      dnsChallenge:
        provider: exec
        delayBeforeCheck: 0

dynamic_conf.yml

http:
  routers:
    router0:
      entyPoints:
      - web
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && PathPrefix(`/nc`)"
      middlewares:
      - redirect
    router1:
      entyPoints:
      - web-secure
      service: nextcloud
      rule: "Host(`raspberrypi.mydomain.example.com`) && PathPrefix(`/nc`)"
      middlewares:
      - removeServiceSelector
      - ncHeader
      tls:
        certResolver: sample
    router2:
      entyPoints:
      - web
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`)  && PathPrefix(`/ad`)"
      middlewares:
      - redirect
    router3:
      entyPoints:
      - web-secure
      service: adminer
      rule: "Host(`raspberrypi.mydomain.example.com`)  && PathPrefix(`/ad`)"
      middlewares:
      - removeServiceSelector
      tls:
        certResolver: sample

  services:
    nextcloud:
      loadBalancer:
        servers:
          - url: "http://raspberrypi.mydomain.example.com:80/"
    adminer:
      loadBalancer:
        servers:
          - url: "http://raspberrypi.mydomain.example.com:8080/"

  middlewares:
    ncHeader:
      headers:
        customResponseHeaders:
          stsPreload: true
          stsSeconds: 15552000
    removeServiceSelector:
      stripPrefix:
        prefixes:
          - "/nc"
          - "/ad"
        forceSlash: false
    redirect:
      redirectScheme:
        scheme: https

nextcloud config.php

<?php
$CONFIG = array (
  'htaccess.RewriteBase' => '/nc',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/html/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/html/custom_apps',
      'url' => '/custom_apps',
      'writable' => true,
    ),
  ),
  'instanceid' => 'asdfasdfadsf',
  'passwordsalt' => 'asdfasdfasdfasdfasdfasdfasdfas',
  'secret' => 'asdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdfasdf',
  'trusted_domains' =>
  array (
    0 => 'nextcloud:80',
    1 => 'raspberrypi.mydomain.example.com',
  ),
  'trusted_proxies' => ['traefik'],
  'overwrite.cli.url' => 'https://raspberrypi.mydomain.example.com/nc',
  'overwritehost' => 'raspberrypi.mydomain.example.com',
  'overwritewebroot' => '/nc',
  'overwriteprotocol' => 'https',
  'datadirectory' => '/var/www/html/data',
  'dbtype' => 'mysql',
  'version' => '16.0.5.1',
  'dbname' => 'nextcloud',
  'dbhost' => 'db',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'asdfasdfasdf',
  'installed' => true,
  'maintenance' => false,
  'theme' => '',
  'loglevel' => 0,
  'mysql.utf8mb4' => true,
);