Traefik v3 cannot get TLS passthrough to work

fracture · June 17, 2024, 1:15am

I am starting with a fresh setup based on my old v2 config, and starting very simple - I want Traefik to proxypass all traffic to another remote host (technically another Traefik box downstream). I cannot for the life of me get this to work, and I'm guessing it's a syntax thing. Note that this Traefik instance receives all internet traffic via Cloudflare, thus the trusted IPs block.

I do not want to terminate TLS on anything except the Traefik dashboard; I want all other traffic to come in on 443 and get passed on to the external host.

If I'm able to get this to work, I am additionally curious if I can get HostSNI to work with specific domains + wildcards, and if so, do I need this Traefik host to be able to pull certs for all inbound domains?

Here's my config:

/traefik/docker-compose.yml

version: '3'
services:
  cloud-reverse-proxy:
    image: traefik:v3.0
    container_name: traefik
    env_file: .env
    ports:
      - '443:443'
      - '8080:8080'
    volumes:
      - /traefik/traefik.yml:/etc/traefik/traefik.yml
      - /traefik/letsencrypt:/letsencrypt

/traefik/traefik.ym

version: '3'
global:
  checkNewVersion: true
log:
  level: DEBUG
entryPoints:
  websecure:
    address: ':443'
    proxyProtocol:
      trustedIPs:
        - 173.245.48.0/20
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 141.101.64.0/18
        - 108.162.192.0/18
        - 190.93.240.0/20
        - 188.114.96.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 162.158.0.0/15
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 172.64.0.0/13
        - 131.0.72.0/22
        - '2400:cb00::/32'
        - '2606:4700::/32'
        - '2803:f800::/32'
        - '2405:b500::/32'
        - '2405:8100::/32'
        - '2a06:98c0::/29'
        - '2c0f:f248::/32'
  dashboard:
    address: ':8080'
api:
    dashboard: true
tcp:
  routers:
    passeverything:
      rule: HostSNI(`*`)
      entrypoints: websecure
      service: front
      tls:
        passthrough: true
  services:
    front:
      loadBalancer:
        servers:
          - address: '10.4.0.100:443'
http:
  routers:
    dashboard:
      rule: Host(`cloudproxy.mydomain.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
      entrypoints: dashboard
      service: api@internal
      tls: 
        certresolver: myresolver
      middlewares:
        - dashauth
  middlewares:
    dashauth:
      basicAuth:
        users:
          - "USER:PASSWORD"
certificatesResolvers:
  myresolver:
    acme:
      email: postmaster@mydomain.com
      storage: /letsencrypt/acme.json
      dnschallenge:
        provider: cloudflare
        delaybeforecheck: 0

/traefik/.env

CF_DNS_API_TOKEN=HERESMYTOKEN

When I try to access an address that should get forwarded, I get this in the Traefik debug logs:

traefik  | 2024-06-17T01:32:17Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default
traefik  | 2024-06-17T01:33:30Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "thisshouldgetpassthrough.mydomain.com"
traefik  | 2024-06-17T01:33:30Z DBG log/log.go:245 > http: TLS handshake error from 172.70.231.30:55302: remote error: tls: unknown certificate authority
traefik  | 2024-06-17T01:33:30Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "thisshouldgetpassthrough.mydomain.com"
traefik  | 2024-06-17T01:33:30Z DBG log/log.go:245 > http: TLS handshake error from 162.158.159.104:18308: remote error: tls: unknown certificate authority

and in the browser, I get an origin cert error from Cloudflare.

If I try to access a direct URL, like the dashboard, I get the same log rows as above, and a 404 in the browser.

bluepuma77 · June 17, 2024, 4:51am

I think with this you enable TLS in Traefik. Then Traefik needs a TLS cert. If you don’t have a cert and just want to pass the traffic untouched, just remove it, it’s a TCP router anyway.

Note that Host() only works on http routers, not on tcp routers.

fracture · June 17, 2024, 10:49am

I did try that, too, but get the same errors if I remove those lines, and even if I explicitly set tls to false:

tcp:
  routers:
    passeverything:
      entrypoints: websecure
      service: front
      tls: false

as for the passthrough flag and Host - I'm using HostSNI just as in the documentation example here - Traefik Routers Documentation - Traefik

bluepuma77 · June 17, 2024, 3:37pm

Sorry, didn’t see the http section header.

It seems you mix static and dynamic config. Dynamic config, like routers and services, need to be placed in a separate file (or declared using labels), loaded via providers.file in static config (traefik.yml).

fracture · June 18, 2024, 3:09am

Boom, thank you, that was it! Here's the final working config:

/traefik/docker-compose.yml

version: '3'
services:
  cloud-reverse-proxy:
    image: traefik:v3.0
    container_name: traefik
    env_file: .env
    ports:
      - '${BIND_ADDRESS_PUBLIC}:443:443'
      - '${BIND_ADDRESS_PRIVATE}:8080:8080'
    volumes:
      - /traefik/traefik.yml:/etc/traefik/traefik.yml
      - /traefik/dynamic:/etc/traefik/dynamic/
      - /traefik/letsencrypt:/letsencrypt

/traefik/traefik.yml

version: '3'
global:
  checkNewVersion: true
entryPoints:
  websecure:
    address: ':443'
    proxyProtocol:
      trustedIPs:
        - 173.245.48.0/20
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 141.101.64.0/18
        - 108.162.192.0/18
        - 190.93.240.0/20
        - 188.114.96.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 162.158.0.0/15
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 172.64.0.0/13
        - 131.0.72.0/22
        - '2400:cb00::/32'
        - '2606:4700::/32'
        - '2803:f800::/32'
        - '2405:b500::/32'
        - '2405:8100::/32'
        - '2a06:98c0::/29'
        - '2c0f:f248::/32'
  dashboard:
    address: ':8080'
api:
    dashboard: true
providers:
  file:
    directory: /etc/traefik/dynamic
certificatesResolvers:
  myresolver:
    acme:
      email: postmaster@mydomain.com
      storage: /letsencrypt/acme.json
      dnschallenge:
        provider: cloudflare
        delaybeforecheck: 0

/traefik/dynamic/dynamic.yml

version: '3'
tcp:
  routers:
    passeverything:
      rule: HostSNI(`*`)
      entrypoints: websecure
      service: front
      tls:
        passthrough: true
  services:
    front:
      loadBalancer:
        servers:
          - address: 'z.z.z.z:443'

http:
  routers:
    dashboard:
      rule: Host(`cloudproxy.mydomain.com`)
      entrypoints: dashboard
      service: api@internal
      tls: 
        certresolver: myresolver
      middlewares:
        - dashauth
  middlewares:
    dashauth:
      basicAuth:
        users:
          - "user:pw"

/traefik/.env

CF_DNS_API_TOKEN=TOKEN
BIND_ADDRESS_PUBLIC=x.x.x.x
BIND_ADDRESS_PRIVATE=y.y.y.y

system · June 21, 2024, 3:10am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Reverse proxy for external apache server Traefik v2 docker-swarm	2	2197	November 4, 2021
TLS passthrough for HTTP router Traefik v2 docker , file	9	7514	May 10, 2023
Traefik 3 + NGINX Proxy Manager Traefik v3 (latest) docker	10	62	November 10, 2024
Traefik through cloudflare tunnel Traefik v3 (latest) docker , letsencrypt-acme	2	210	October 14, 2024
Docker Traefik to K3S Traefik subdomain reverse proxy Traefik v2 rancher , docker , letsencrypt-acme , tcp	1	1037	January 31, 2023

Traefik v3 cannot get TLS passthrough to work

Related topics