Tcp router with tls and default certs

Traefik Traefik v2 (latest)
1

I am trying to use traefik to terminate a DNS-over-TLS server. Current using unbound because I was already using it.

I cannot get traefik to serve the proper certificate for this entrypoint. It only seems to serve the default traefik cert which results in an error due to mismatch. This post seems similiar, but not quite and answer does't seem to help me.

relevant docker-compose.yml sections

services:
  unbound:
    container_name: unbound
    image: mvance/unbound:latest
    hostname: unbound
    networks:
      - dot-net
    ports:
      - 53:53/tcp
      - 53:53/udp
    environment:
      TZ: $TZ
    volumes:
      - ./unbound/unbound.conf:/opt/unbound/etc/unbound/unbound.conf
      - ./unbound/unbound.log:/opt/unbound/etc/unbound/unbound.log
    restart: unless-stopped
    labels:
      - traefik.enable=true
      - traefik.tcp.routers.dnsovertls.rule=HostSNI(`dot.$DOMAINNAME`)
      - traefik.tcp.routers.dnsovertls.entrypoints=dot
      - traefik.tcp.routers.dnsovertls.tls.certResolver=cloudflare
      - traefik.tcp.routers.dnsovertls.service=unbound
      - traefik.tcp.services.unbound.loadbalancer.server.port=53

  traefik:
    image: traefik:v3.0.0-beta5
    container_name: traefik
    logging:
      options:
        max-size: "3m"
        max-file: "3"
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - dot-net
    ports:
      - 80:80
      - 443:443
      - 853:853
      - 8082:8080
    secrets:
      - cf_dns_api
      - cf_zone_api_all
      - cf_api_email
    environment:
      - CF_ZONE_API_TOKEN_FILE=/run/secrets/cf_zone_api_all
      - CF_DNS_API_TOKEN_FILE=/run/secrets/cf_dns_api
      - CF_API_EMAIL_FILE=/run/secrets/cf_api_email
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik:/etc/traefik
    command:
      - --api.dashboard=true
      - --api.insecure=true
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=dot-net
      - --providers.file.filename=/etc/traefik/dynamic-conf.yml
      - --log.level=DEBUG #DEBUG INFO WARN ERROR PANIC FATAL
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.dot.address=:853
      - --certificatesresolvers.cloudflare.acme.storage=/etc/traefik/acme.json
      - --certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.cloudflare.acme.dnschallenge.resolvers=1.1.1.1:53,1.0.0.1:53

dynamic-conf.yml

tls:
  options:
    default:
      alpnProtocols:
        - http/1.1
        - h2
        - dot
        - acme-tls/1

Plain DNS resolution works:
dig +tcp @192.168.2.6 example.com results in NOERROR and returned IP

dns-over-tls results in error:

kdig -d @192.168.2.6 +tls-ca +tls-host=dot.mydomain.com example.com
;; DEBUG: Querying for owner(example.com.), class(1), type(1), server(192.168.2.6), port(853), protocol(TCP)
;; DEBUG: TLS, imported 137 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, CN=TRAEFIK DEFAULT CERT
;; DEBUG:      SHA-256 PIN: Q3IXFGc1HQA5pgq462PVzUeHUT1op8utxDFNSy62jk4=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is NOT trusted. The certificate issuer is unknown. The name in the certificate does not match the expected.
;; WARNING: TLS, handshake failed (Error in the certificate.)
;; ERROR: failed to query server 192.168.2.6@853(TCP)

and in traefik I see:

2024-02-05T12:32:32-05:00 DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:220 > Serving default certificate for request: ""
2024-02-05T12:32:32-05:00 DBG log/log.go:245 > http: TLS handshake error from 192.168.1.10:50785: read tcp 172.24.0.3:853->192.168.1.10:50785: read: connection reset by peer

Seems it sends the default certificate because there is no host name sent. Am I doing something wrong with my router?

I am able to get around this if I set the default certificate as my ACME certificate, because then the 'default' matches what my request is expecting, but I was hoping to avoid this.

2

Does unbound show up in Traefik dashboard? How do you create the Docker network?

3

bluepuma77

Does unbound show up in Traefik dashboard? How do you create the Docker network?

TCP router looks ok. See screenshots below.
My simple docker network is setup in my compose:

version: '3.9'

networks:
  dot-net:
    name: dot-net

image
image965×858 32.8 KB

image
image944×408 17.8 KB

I have no issues browsing a sample 'whoami' container through traefik as seen below. The proper LE cert is used.

  whoamilocal:
    image: traefik/whoami:v1.10.1
    container_name: whoamilocal
    restart: unless-stopped
    networks:
      - dot-net
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoamilocal.rule=Host(`whoami.$DOMAINNAME`)
      - traefik.http.routers.whoamilocal.entrypoints=websecure

Are TCP routers supposed to recieve the HostSNI? Perhaps it is a problem with the way I'm testing my request. I tried doing a router with HostSNI(*) as I saw in some examples, but I guess that won't work with TLS connection.

4

@bluepuma77
got any other ideas for this issue?

5

When using modern TLS, the domain should be included in the TLS request and HostSNI() should be able to read it.

Even when using with *, Traefik should still serve an existing matching cert, only then fall back to default.

What’s the default port for DNS-over-TLS?

Do you see a cert for dot.<domain> in acme.json file?

Compare to simple Traefik TCP example.

Are you sure kdig command is correct?

Note that you don’t need ports: on unbound, Traefik should use the Docker network, within all ports are open.