Traefik and Pihole in Docker Swarm

Traefik Traefik v2
,
1

Hi all,

I've experimenting with a docker-swarm setup that runs, among other things, Traefik (on manager node) and Pi-Hole (on manager node and worker node). There are 3 working configurations, with drawbacks, and I'm hoping someone can illuminate me on potential solutions!

expose 53 over ingress network
Running pihole as-is, using port 53 published over the swarm ingress network, makes sure that DNS queries are load balanced by docker. However, Pi-Hole cannot differentiate client IP's anymore (pihole recommends running host-exposed ports to accomplish this), so each request is logged as originated from the docker-swarm manager node.

load balance 53 over traefik
Having traefik listen on tcp/udp 53 and then use the docker-swarm integration (add traefik labels on the pihole docker config for tcp, udp and http) works OK as well: traffic is load balanced. However, now all calls seems to originate from 'traefik.container.name'. Same problem, different layer.

expose ports on host
Expose 53 on the host port. This allows the proper resolve of each client IP, however, load balancing is not managed by docker anymore (as the ingress network is bypassed). I guess I could try load balancing through Traefik by manually setting the different host IP's, but that would not be taking into account newly created nodes, removed containers, ..., so it feels like a less desirable approach.

So, getting a fresh idea as to getting pihole to behave in a docker swarm, potentially combined with Traefik, would be great. This would mean 1) automatically load balancing to the various pihole containers 2) resolving client ip's properly.

2

Hey CountZukula, have you been able to find a solution for this?

I have the same issue and would love to run my Pi-Hole behind traefik with proper client IPs :slight_smile:

3

I'm afraid not, running in 'ingress' mode now. So pihole is published on the swarm, exposing port 53 over udp/tcp, in mode "ingress". This results in pihole showing a single origin ip.

I'm still interested in finding a proper solution though, once I switch machines around and start upgrading services I'll have a look at it again. For now, can't be bothered :).

services:
  pihole:
    image: pihole/pihole:latest
    hostname: '{{.Node.Hostname}}'
    ports:
      - target: 53
        published: 53
        protocol: tcp
        mode: ingress 
      - target: 53
        published: 53
        protocol: udp
        mode: ingress
    environment:
      TZ: 'Europe/Brussels'
      WEBPASSWORD: 'xxx'
      # this supposedly reverts to previous behaviour to support cap_add
      FTL_CMD: 'debug'
      DNSMASQ_LISTENING: 'all'
    volumes:
       - '/srv/pihole/etc-pihole/:/etc/pihole/'
       - '/srv/pihole/etc-dnsmasq.d/:/etc/dnsmasq.d/'
    dns:
      - 127.0.0.1
      - 9.9.9.9
    deploy:
      replicas: 1
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
        window: 120s
      labels:
        - "traefik.enable=true"
        - "traefik.http.services.pihole.loadbalancer.server.port=80"
        - "traefik.http.services.pihole.loadbalancer.sticky.cookie=true"
        - "traefik.http.routers.pihole.rule=Host(`pihole.server`)"
        - "traefik.http.routers.pihole.service=pihole"
        - "traefik.http.routers.pihole.entrypoints=web"
      placement:
        constraints:
          - "node.labels.dns==true"

networks:
  default:
    external: true
    name: frontend
4

Hi @CountZukula , @Ota

You can load-balance pihole with traefik, but you will still have to expose the ports for traefik using host networking.

The remaining issue with this as the clients need the ip of the traefik node. By default a linux hosts resolv.conf allows a max of 3 resolvers. I'm not up to date with how resolvconf/systemd-resolved/windows/mac would handle an offline dns server, but I'm relatively confident they allow 3 (whether or not a given router /dhcp server does is another question).

So if you have 3 nodes(minimum viable swarm) where traefik will run just adding those ip's as the resolvers may work.

Beyond that another solution in front of, or in conjunction with traefik is needed. I'm thinking of foating ip with keepalived/corosync.

5

Hey @cakiwi,

when running network: host, how is traefik still able to proxy to other services? Afaik with network: host it is not possible to join other bridge networks.

6

You can do it port by port.

7

Ah, didn't know that. Went ahead and tested it, but it seems that still, all clients show up as one host in the query log. Did I miss anything?

image
image1233×900 114 KB

My Traefik and Pi-Hole docker configuration is as follows

#
# Traefik
#
version: '3.3'

services:
  traefik:
    image: 'traefik:v2.3'
    restart: 'unless-stopped'
    deploy:
      placement:
        constraints:
          - node.role == manager
      labels:
        - 'traefik.enable=true'
        - 'traefik.http.routers.api.rule=Host(`traefik.xxx`)'
        - 'traefik.http.routers.api.service=api@internal'
        - 'traefik.http.routers.api.entrypoints=web'
        - 'traefik.http.services.api.loadbalancer.server.port=80'
    command:
      # - '--log.level=DEBUG'
      - '--api.insecure=true'
      - '--providers.docker.swarmMode=true'
      - '--providers.docker.exposedbydefault=false'
      - '--providers.docker.network=traefik'
      - '--entrypoints.web.address=:80'
      - '--entrypoints.websecure.address=:433'
      - '--entrypoints.portainer.address=:9000'
      - '--entrypoints.dns.address=:53'
      - '--entrypoints.dns_udp.address=:53/udp'
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: ingress 
      - target: 433
        published: 433
        protocol: tcp
        mode: ingress
      - target: 9000
        published: 9000
        protocol: tcp
        mode: ingress
      - target: 53
        published: 53
        protocol: tcp
        mode: host
      - target: 53
        published: 53
        protocol: udp
        mode: host
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock:ro'
    networks:
      - traefik

  whoami:
    image: 'traefik/whoami'
    deploy:
      labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.whoami.rule=Host(`whoami.xxx`)'
      - 'traefik.http.routers.whoami.entrypoints=web'
      - 'traefik.http.services.whoami.loadbalancer.server.port=80'
    networks:
      - traefik

networks:
  traefik:
    external: true

#
# Pi-Hole
#

version: '3.4'

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    image: 'jacklul/pihole:latest'
    restart: 'unless-stopped'
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.pihole.rule=Host(`pihole.xxx`)"
        - "traefik.http.routers.pihole.entrypoints=web"
        - "traefik.http.services.pihole.loadbalancer.server.port=80"
        
        - "traefik.tcp.routers.dns.rule=HostSNI(`*`)"
        - "traefik.tcp.routers.dns.entrypoints=dns"
        - "traefik.tcp.routers.dns.service=pihole"
        - "traefik.tcp.services.pihole.loadbalancer.server.port=53"

        - "traefik.udp.routers.dns_udp.entrypoints=dns_udp"
        - "traefik.udp.routers.dns_udp.service=pihole"
        - "traefik.udp.services.pihole.loadbalancer.server.port=53"
    environment:
      TZ: 'Europe/Berlin'
      DNS1: '1.1.1.1'
      WEBPASSWORD_FILE: '/run/secrets/pihole_webpassword'
      ServerIP: '192.168.20.93'
      VIRTUAL_HOST: 'pihole.xxx'
      DNS_FQDN_REQUIRED: 'true'
      DNS_BOGUS_PRIV: 'true'
    volumes:
      - 'pihole_etc-pihole:/etc/pihole/'
      - 'pihole_etc-dnsmasq.d:/etc/dnsmasq.d/'
      - 'pihole_etc-pihole-updatelists:/etc/pihole-updatelists/'
    networks:
       traefik:
    dns:
      - 127.0.0.1
      - 1.1.1.1
    secrets:
        - pihole_webpassword
    configs:
      - source: pihole-FTL.conf
        target: /etc/pihole/pihole-FTL.conf
      - source: 99-conditional-forwarding.conf
        target: /etc/dnsmasq.d/99-conditional-forwarding.conf

volumes:
    pihole_etc-pihole:
        external: true
    pihole_etc-dnsmasq.d:
        external: true
    pihole_etc-pihole-updatelists:
        external: true

secrets:
  pihole_webpassword:
    external: true

configs:
  pihole-FTL.conf:
    external: true
  99-conditional-forwarding.conf:
    external: true

networks:
  traefik:
    external: true
8

Sorry I could be wrong on this. I'll see if I can test the same today.

Expose ports.

Note

Port mapping is incompatible with network_mode: host

9

I am trying to solve this as well. Has there been any progress from anyone on this thread?

I suppose you could use macvlan and attach pihole with only port 53/tcp/udp published to assign an ip from the router's subnet. This would then be the DNS server IP instead of traefik. The only thing traefik would load balance are port 80 requests to pihole. You would lose out the full load balancing feature. I think the other issue would be to allocate IP's on the router for all the instances. Possibly ip conflict if there isn't a proper rule set to terminate an ip lease.