DNS-over-TLS almost working

Traefik Traefik v2
,
1

Hi!

I'm trying to implement DNS-over-TLS using pihole behind Traefik.

My setup is almost working, but despite Traefik not setup to passthrough the TLS to the TCP service the service is receiving encrypted data. I found that out doing a tcpdump of the port 53 on my pihole docker interface.

If I do a dig @pihole_docker_ip_address linux.org I can see in the tcpdump that the packets are in clear text and I get a result to my query.

When I try to connect my Android client to my DoT setup Traefik throws me an error

"Error during connection: readfrom tcp 172.18.0.2:49132->172.18.0.4:53: remote error: tls: expired certificate"

172.18.0.2 being Traefik and 172.18.0.4 being the pihole DNS service.
tcpdump show encrypted packets coming from Traefik to the dns interface.

All my certs are valid and working, I'm using CloudFlare API.
Traefik WebUI show that my router is not in passthrough but yet traffic seems to be passthrough encrypted. How can I force Traefik to terminate the TLS connection and pass the packets decrypted to my service?

SjVsNWK1
SjVsNWK11427×842 50.3 KB

2

Thanks for your interest in Traefik!

The documentation has more info about TLS termination.

See this YAML example:

## Dynamic configuration
tcp:
  routers:
    Router-1:
      rule: "HostSNI(`foo-domain`)"
      service: service-id
      # will terminate the TLS request by default
      tls: {}
3

Thank you for your reply.

I'm using labels in docker compose and my router does need to serve a tls certificate. Here what I have regarding this router at the moment in my docker compose.

services:
  pihole:
    ...
    labels:
      ...
      # DNS-over-TLS
      - "traefik.tcp.routers.pihole-dot.rule=HostSNI(`foo-domain`)"
      - "traefik.tcp.routers.pihole-dot.entrypoints=dnsovertls"
      - "traefik.tcp.routers.pihole-dot.tls.certresolver=letsencrypt"
      - "traefik.tcp.routers.pihole-dot.service=pihole-tcp"
      - "traefik.tcp.services.pihole-tcp.loadbalancer.server.port=53"
      ...

Entrypoint dnsovertls is defined as follow in my traefik config file:

entryPoints:
  dnsovertls:
    address: ":853"
4

Can you share your full Traefik static and dynamic config, and docker-compose.yml if used?

Enable Traefik debug log and check for "error".

5

Here are my configs! Thanks again for the interest you have in my problem, I'm sure a lot of guys would like to be able to do the same as me.

docker-compose.yml for Traefik 
version: "3.8"

services:
  traefik:
    image: "traefik:latest"
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - "no-new-privileges:true"
    networks:
      - proxy
    ports:
      - "80:80"
      - "443:443"
      - "853:853"
    volumes:
      - "/etc/localtime:/etc/localtime:ro"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./traefik-data/traefik.yml:/traefik.yml:ro"
      - "./traefik-data/acme.json:/acme.json"
      - "./traefik-data/configurations:/configurations"
    environment:
      - "CF_DNS_API_TOKEN=<cf_token>"
    labels:
      - traefik.enable=true
      - traefik.docker.network=proxy
      - traefik.http.routers.traefik-secure.entrypoints=websecure
      - traefik.http.routers.traefik-secure.rule=Host(`traefik.example.com`)
      - traefik.http.routers.traefik-secure.service=api@internal
      - traefik.http.routers.traefik-secure.middlewares=user-auth@file
traefik.yml 
api:
  dashboard: true

log:
  level: INFO
  format: common

#accesslog:
#  format: common

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
  websecure:
    address: ":443"
    http:
      middlewares:
        - secureHeaders@file
      tls:
        certResolver: letsencrypt
  dnsovertls:
    address: ":853"

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: /configurations/dynamic.yml

certificatesResolvers:
  letsencrypt:
    acme:
      email: me@privacy.net
      storage: acme.json
      dnschallenge:
        provider: cloudflare
dynamic.yml 
# Dynamic configuration
http:
  middlewares:
    secureHeaders:
      headers:
        sslRedirect: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 31536000
    user-auth:
      basicAuth:
        users:
          - "me:password"

tls:
  stores:
    default:
      defaultGeneratedCert:
        resolver: letsencrypt
        domain:
          main: example.com
          sans:
            - dns.example.com
  options:
    default:
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
      minVersion: VersionTLS12
docker-compose.yml for pihole 
version: "3.9"

networks:
  proxy:
    external: true

services:
  pihole:
    image: pihole/pihole:latest
    container_name: pihole
    hostname: dns.example.com
    networks:
      - proxy
    environment:
      PUID: '1001'
      PGID: '1001'
      TZ: 'America/Toronto'
      WEBPASSWORD: 'best_strong_password'
    volumes:
      - './etc-pihole/:/etc/pihole/'
      - './etc-dnsmasq.d/:/etc/dnsmasq.d/'
    dns:
      - 1.0.0.1
      - 1.1.1.1
    expose:
      - 80
      - 53
      - 53/udp
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"

      # web interface
      - "traefik.http.routers.pihole-gui.rule=Host(`dns.example.com`)"
      - "traefik.http.routers.pihole-gui.entrypoints=websecure"
      - "traefik.http.routers.pihole-gui.middlewares=pihole_PiholeChain"
      - "traefik.http.services.pihole-gui.loadbalancer.server.port=80"

      # make sure '/admin' is there
      - "traefik.http.middlewares.pihole_AddAdminPath.replacepathregex.regex=^/((?i:(admin)/{0,1}|.{0})(.*))"
      - "traefik.http.middlewares.pihole_AddAdminPath.replacepathregex.replacement=/admin/$$3"
      - "traefik.http.middlewares.pihole_PiholeChain.chain.middlewares=pihole_AddAdminPath,secureHeaders@file"

      # DNS-over-TLS
      - "traefik.tcp.routers.pihole-dot.rule=HostSNI(`dns.example.com`)"
      - "traefik.tcp.routers.pihole-dot.entrypoints=dnsovertls"
      - "traefik.tcp.routers.pihole-dot.tls.certresolver=letsencrypt"
      - "traefik.tcp.routers.pihole-dot.tls=true"
      - "traefik.tcp.routers.pihole-dot.service=pihole-tcp"
      - "traefik.tcp.services.pihole-tcp.loadbalancer.server.port=53"

The error I'm getting is this one:
traefik | time="2023-01-18T03:24:32Z" level=error msg="Error during connection: readfrom tcp 172.18.0.2:38970->172.18.0.4:53: remote error: tls: expired certificate"

It appear when I try to setup DoT on my Android as a client. I get no other error than that one... Very non descriptive.

6

Did you manage to solve it?

I have it working and these are my labels for DoT for the Adguard container:

# DNS-over-TLS
  - "traefik.tcp.routers.dot.rule=HostSNI(`adguard.example.com`)"
  - "traefik.tcp.routers.dot.entrypoints=dot"
  - "traefik.tcp.routers.dot.tls.passthrough=true"
  - "traefik.tcp.routers.dot.service=dot"
  - "traefik.tcp.services.dot.loadbalancer.server.port=853"

I also had to add this to my TLS options:

alpnProtocols:
- http/1.1
- h2
- acme-tls/1
- dot

1 Like