Http: TLS handshake error from : remote error: tls: unknown certificate"

Traefik Traefik v2
1

Hi,

I'm trying to setup traefik for to redirect to multiple VMs in the same network.

Traefik is running in a docker VM (192.168.1.231).
I have services in different docker VM instances (192.168.1.x)

I have configured several services, none of them are working. However, I have the feeling I have the same problem on all services. So we are going to focus on grafana.domain.net.

Grafana is working at http://192.168.1.223:3001 so we can isolate the problem in the traefik config.

Attached below is the configuration and the log output.

docker-compose.yml

version: '3'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      proxy:
        #ipv4_address: 172.27.0.3
    command:
      - "--entrypoints.redis.address=:6379" # Redis endpoint.
    ports:
      - 80:80
      - 443:443
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data/traefik.yml:/traefik.yml:ro
      - ./data/traefik-dynamic.yml:/traefik-dynamic.yml:ro
      - ./data/acme.json:/acme.json
      - /var/log/containers/traefik/log.log:/log.log
      - ./certs:/certs
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.domain.net`)"
      - "traefik.http.routers.traefik-secure.entrypoints=websecure"
      #- "traefik.http.routers.traefik-secure.middlewares=traefik-auth@file"
      #- "traefik.http.middlewares.traefik-auth.basicauth.users=mbastida:$$apr1$$qqDknjl0$$U6XvKDZp.tNWERMXPvrn6."
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=lets-encrypt"
      - "traefik.http.routers.traefik-secure.service=api@internal"

networks:
  proxy:
    external: true

traefik.yml

api:
  dashboard: true

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
          
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: /traefik-dynamic.yml
    watch: true
      
certificatesResolvers:
  lets-encrypt:
    acme:
      email: 603mbastida@gmail.com
      storage: acme.json
      httpChallenge:
        entryPoint: web

log:
  filePath: "/log.log"
  level: DEBUG

traefik-dynamic.yml

http:
  middlewares:
    traefik-auth:
      basicAuth:
        users:
          - "mbastida:$2y$05$MicNnLH1joV9sPHZKLe89OQHy3ArbRaoDbZk2KJTvFNPGb.qSGJQu"
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https
  routers:
    proxmox:
      entryPoints:
        - "websecure"
      rule: "Host(`proxmox.domain.net`)"
      middlewares:
        - default-headers
      tls: {}
      service: proxmox
    homeassistant:
      # For Homeassistant config, check: https://www.home-assistant.io/integrations/http/#reverse-proxies
      # This relies on Homeassistant using http. No certs are needed in the Homeassistant config.
      entryPoints:
        - "websecure"
      rule: "Host(`home.domain.net`)"
      middlewares:
        - default-headers
      tls: {}
      service: ha
    grafana:
      entryPoints:
        - "websecure"
      rule: "Host(`grafana.domain.net`)"
      middlewares:
        - default-headers
      tls: {}
      service: grafana
    influx:
      entryPoints:
        - "websecure"
      rule: "Host(`influxdb2.domain.net`)"
      middlewares:
        - default-headers
      tls: {}
      service: influx
  services:
    proxmox:
      loadBalancer:
        servers:
          - url: "https://192.168.1.231:8006"
    ha:
      loadBalancer:
        servers:
          - url: "https://192.168.1.201:8123"
    grafana:
      loadBalancer:
        servers:
          - url: "http://192.168.1.223:3001"
    influx:
      loadBalancer:
        servers:
          - url: "https://192.168.1.223:8089"
    nodered:
      loadBalancer:
        servers:
          - url: "https://192.168.1.201:1880"

time="2023-07-31T20:30:26+02:00" level=info msg="Traefik version 2.10.4 built on 2023-07-24T16:29:02Z"
time="2023-07-31T20:30:26+02:00" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"websecure\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"watch\":true,\"filename\":\"/traefik-dynamic.yml\"}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"filePath\":\"/log.log\",\"format\":\"common\"},\"certificatesResolvers\":{\"lets-encrypt\":{\"acme\":{\"email\":\"603mbastida@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"web\"}}}}}"
time="2023-07-31T20:30:26+02:00" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
time="2023-07-31T20:30:26+02:00" level=debug msg="Starting TCP Server" entryPointName=websecure
time="2023-07-31T20:30:26+02:00" level=debug msg="Starting TCP Server" entryPointName=web
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider *file.Provider"
time="2023-07-31T20:30:26+02:00" level=debug msg="*file.Provider provider configuration: {\"watch\":true,\"filename\":\"/traefik-dynamic.yml\"}"
time="2023-07-31T20:30:26+02:00" level=error msg="Error while building configuration (for the first time): field not found, node: stores" providerName=file
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider *traefik.Provider"
time="2023-07-31T20:30:26+02:00" level=debug msg="*traefik.Provider provider configuration: {}"
time="2023-07-31T20:30:26+02:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"acme-http\":{\"entryPoints\":[\"web\"],\"service\":\"acme-http@internal\",\"rule\":\"PathPrefix(`/.well-known/acme-challenge/`)\",\"priority\":2147483647},\"web-to-websecure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-websecure\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"acme-http\":{},\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"redirect-web-to-websecure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-07-31T20:30:26+02:00" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider *docker.Provider"
time="2023-07-31T20:30:26+02:00" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"}"
time="2023-07-31T20:30:26+02:00" level=info msg="Starting provider *acme.Provider"
time="2023-07-31T20:30:26+02:00" level=debug msg="*acme.Provider provider configuration: {\"email\":\"603mbastida@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"web\"},\"ResolverName\":\"lets-encrypt\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
time="2023-07-31T20:30:26+02:00" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" providerName=lets-encrypt.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-07-31T20:30:26+02:00" level=info msg="Testing certificate renew..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=lets-encrypt.acme
time="2023-07-31T20:30:26+02:00" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=lets-encrypt.acme
time="2023-07-31T20:30:26+02:00" level=debug msg="Provider connection established with docker 24.0.5 (API 1.43)" providerName=docker
time="2023-07-31T20:30:26+02:00" level=debug msg="Filtering disabled container" providerName=docker container=pihole-pihole-34a5322c7214b8ee4f9707315c7cc8b129e29c2e2a27b8b736c42a8fb9144dc6
time="2023-07-31T20:30:26+02:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"traefik-secure\":{\"entryPoints\":[\"websecure\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.domain.net`)\",\"tls\":{\"certResolver\":\"lets-encrypt\"}}},\"services\":{\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.19.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-07-31T20:30:26+02:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-07-31T20:30:26+02:00" level=debug msg="Added outgoing tracing middleware acme-http@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=web routerName=acme-http@internal
time="2023-07-31T20:30:26+02:00" level=debug msg="Added outgoing tracing middleware noop@internal" routerName=web-to-websecure@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=web
time="2023-07-31T20:30:26+02:00" level=debug msg="Creating middleware" middlewareType=RedirectScheme entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal
time="2023-07-31T20:30:26+02:00" level=debug msg="Setting up redirection to https 443" middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme entryPointName=web routerName=web-to-websecure@internal
time="2023-07-31T20:30:26+02:00" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
time="2023-07-31T20:30:26+02:00" level=debug msg="Adding certificate for domain(s) traefik.domain.net"
time="2023-07-31T20:30:27+02:00" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-07-31T20:30:27+02:00" level=debug msg="Added outgoing tracing middleware noop@internal" middlewareType=TracingForwarder entryPointName=web routerName=web-to-websecure@internal middlewareName=tracing
time="2023-07-31T20:30:27+02:00" level=debug msg="Creating middleware" middlewareType=RedirectScheme entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal
time="2023-07-31T20:30:27+02:00" level=debug msg="Setting up redirection to https 443" entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme
time="2023-07-31T20:30:27+02:00" level=debug msg="Added outgoing tracing middleware acme-http@internal" middlewareType=TracingForwarder entryPointName=web routerName=acme-http@internal middlewareName=tracing
time="2023-07-31T20:30:27+02:00" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
time="2023-07-31T20:30:27+02:00" level=debug msg="Added outgoing tracing middleware api@internal" entryPointName=websecure routerName=traefik-secure@docker middlewareName=tracing middlewareType=TracingForwarder
time="2023-07-31T20:30:27+02:00" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=websecure
time="2023-07-31T20:30:27+02:00" level=debug msg="Adding route for traefik.domain.net with TLS options default" entryPointName=websecure
time="2023-07-31T20:30:27+02:00" level=debug msg="Trying to challenge certificate for domain [traefik.domain.net] found in HostSNI rule" providerName=lets-encrypt.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker rule="Host(`traefik.domain.net`)"
time="2023-07-31T20:30:27+02:00" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.domain.net\"]..." rule="Host(`traefik.domain.net`)" providerName=lets-encrypt.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker
time="2023-07-31T20:30:27+02:00" level=debug msg="No ACME certificate generation required for domains [\"traefik.domain.net\"]." routerName=traefik-secure@docker rule="Host(`traefik.domain.net`)" providerName=lets-encrypt.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-07-31T20:31:05+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:09+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:09+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:53658: local error: tls: bad record MAC"
time="2023-07-31T20:31:14+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:14+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51942: remote error: tls: unknown certificate"
time="2023-07-31T20:31:14+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:14+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51943: remote error: tls: unknown certificate"
time="2023-07-31T20:31:15+02:00" level=debug msg="Serving default certificate for request: \"grafana.domain.net\""
time="2023-07-31T20:31:15+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51944: remote error: tls: unknown certificate"
time="2023-07-31T20:31:15+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:15+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51946: remote error: tls: unknown certificate"
time="2023-07-31T20:31:15+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:15+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51947: remote error: tls: unknown certificate"
time="2023-07-31T20:31:18+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:18+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51948: remote error: tls: unknown certificate"
time="2023-07-31T20:31:18+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:18+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51949: remote error: tls: unknown certificate"
time="2023-07-31T20:31:19+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:19+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51950: remote error: tls: unknown certificate"
time="2023-07-31T20:31:22+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:22+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51952: remote error: tls: unknown certificate"
time="2023-07-31T20:31:22+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:22+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51953: remote error: tls: unknown certificate"
time="2023-07-31T20:31:22+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:27+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:27+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51960: remote error: tls: unknown certificate"
time="2023-07-31T20:31:27+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:27+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:51961: remote error: tls: unknown certificate"
time="2023-07-31T20:31:29+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:29+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:59456: local error: tls: bad record MAC"
time="2023-07-31T20:31:54+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:31:57+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:57+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52060: remote error: tls: unknown certificate"
time="2023-07-31T20:31:57+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:31:57+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52061: remote error: tls: unknown certificate"
time="2023-07-31T20:31:59+02:00" level=debug msg="Serving default certificate for request: \"grafana.domain.net\""
time="2023-07-31T20:31:59+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52062: remote error: tls: unknown certificate"
time="2023-07-31T20:31:59+02:00" level=debug msg="Serving default certificate for request: \"grafana.domain.net\""
time="2023-07-31T20:31:59+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52063: remote error: tls: unknown certificate"
time="2023-07-31T20:32:03+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:03+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52064: remote error: tls: unknown certificate"
time="2023-07-31T20:32:03+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:03+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52065: remote error: tls: unknown certificate"
time="2023-07-31T20:32:08+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:32:09+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:32:09+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:47880: local error: tls: bad record MAC"
time="2023-07-31T20:32:09+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:09+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52067: remote error: tls: unknown certificate"
time="2023-07-31T20:32:09+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:09+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52068: remote error: tls: unknown certificate"
time="2023-07-31T20:32:15+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:15+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52070: remote error: tls: unknown certificate"
time="2023-07-31T20:32:15+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:15+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52071: remote error: tls: unknown certificate"
time="2023-07-31T20:32:16+02:00" level=debug msg="Serving default certificate for request: \"grafana.domain.net\""
time="2023-07-31T20:32:16+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52072: remote error: tls: unknown certificate"
time="2023-07-31T20:32:16+02:00" level=debug msg="Serving default certificate for request: \"grafana.domain.net\""
time="2023-07-31T20:32:16+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52073: remote error: tls: unknown certificate"
time="2023-07-31T20:32:21+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:21+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52074: remote error: tls: unknown certificate"
time="2023-07-31T20:32:22+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:22+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52075: remote error: tls: unknown certificate"
time="2023-07-31T20:32:27+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:27+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52078: remote error: tls: unknown certificate"
time="2023-07-31T20:32:28+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:28+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52079: remote error: tls: unknown certificate"
time="2023-07-31T20:32:29+02:00" level=debug msg="Serving default certificate for request: \"influxdb2.domain.net\""
time="2023-07-31T20:32:29+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:38344: local error: tls: bad record MAC"
time="2023-07-31T20:32:33+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:33+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52082: remote error: tls: unknown certificate"
time="2023-07-31T20:32:34+02:00" level=debug msg="Serving default certificate for request: \"home.domain.net\""
time="2023-07-31T20:32:34+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:52083: remote error: tls: unknown certificate"
2

Two mistakes:

  1. You can’t use static config in traefik.yml and command at the same time
  2. You enable TLS with {}, but this is for loaded TLS certs. For LetsEncrypt you need to assign the certresolver

See simple Traefik example.

3

Hi @bluepuma77

For @1 you mean that I can't redirect http to https in traefik.yml? If afirmative is there a way to redirect http to https in a central place without having to specify it in every configuration?

4

And as for @2 I have changed the configuration to:

    grafana:
      entryPoints:
        - "websecure"
      rule: "Host(`grafana.domain.net`)"
      middlewares:
        - default-headers
      tls: 
        stores:
          default:
            defaultGeneratedCert:
              resolver: lets-encrypt
                domain:
                  main: domain.net
                  sans: 
                    - grafana.domain.net
      service: grafana

This should acquire the lets-encrypt certificate right?

5

Just look at simple Traefik example. It has a central redirect on entrypoint web and central LetsEncrypt on entrypoint websecure, all you need is Host() with domain in your router rule.

6

Thanks @bluepuma77
So following your instructions I have ditched the traefik.yml and added the commands to the docker-compose.yml:

version: '3'

services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      proxy:
        #ipv4_address: 172.27.0.3
    command:
      - "--entrypoints.redis.address=:6379" # Redis endpoint.
      - --api.dashboard=true
      - --log.level=DEBUG
      #- --log.filepath=/var/log/traefik.log
      - --accesslog=true
      #- --accesslog.filepath=/var/log/traefik-access.log
      - --providers.docker.network=proxy
      - --providers.docker.exposedByDefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entryPoints.web.http.redirections.entrypoint.scheme=https
      - --entrypoints.websecure.address=:443
      #- --entrypoints.websecure.asDefault=true
      - --entrypoints.websecure.http.tls.certresolver=lets-encrypt
      - --certificatesresolvers.lets-encrypt.acme.email=603mbastida@gmail.com
      - --certificatesresolvers.lets-encrypt.acme.tlschallenge=true
      - --certificatesresolvers.lets-encrypt.acme.storage=/acme.json
    ports:
      - 80:80
      - 443:443
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data/traefik-dynamic.yml:/traefik-dynamic.yml:ro
      - ./data/acme.json:/acme.json
      - /var/log/containers/traefik/log.log:/log.log
      - ./certs:/certs
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.domain.xyz`)"
      - "traefik.http.routers.traefik-secure.entrypoints=websecure"
      #- "traefik.http.routers.traefik-secure.middlewares=traefik-auth@file"
      #- "traefik.http.middlewares.traefik-auth.basicauth.users=mbastida:$$apr1$$qqDknjl0$$U6XvKDZp.tNWERMXPvrn6."
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=lets-encrypt"
      - "traefik.http.routers.traefik-secure.service=api@internal"

networks:
  proxy:
    external: true

The traefik-dynamic.yml has been changed to (I have also reduced to the grafana configuration):

http:
  middlewares:
    traefik-auth:
      basicAuth:
        users:
          - "mbastida:$2y$05$MicNnLH1joV9sPHZKLe89OQHy3ArbRaoDbZk2KJTvFNPGb.qSGJQu"
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https
  routers:
    grafana:
      entryPoints:
        - "websecure"
      rule: "Host(`grafana.domain.xyz`)"
      middlewares:
        - default-headers
      service: grafana
  services:
    grafana:
      loadBalancer:
        servers:
          - url: "http://192.168.1.223:3001"

However, when I start traefik I don't see the grafana certificate being generated and morever in acme.json I only see the traefik.domain certificate.

traefik  | time="2023-08-01T20:22:43+02:00" level=debug msg="Serving default certificate for request: \"grafana.bascomas.xyz\""
traefik  | time="2023-08-01T20:22:43+02:00" level=debug msg="http: TLS handshake error from 192.168.1.1:57404: remote error: tls: unknown certificate"
7

You miss providers.file in command to load your dynamic config file.

1 Like
8

Thanks @bluepuma77, that was the issue.

Grafana is now working.

However, I have tried to add other services and I get 404 error on all of them:

http:
  middlewares:
    traefik-auth:
      basicAuth:
        users:
          - "mbastida:$2y$05$MicNnLH1joV9sPHZKLe89OQHy3ArbRaoDbZk2KJTvFNPGb.qSGJQu"
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https
  routers:
    grafana:
      entryPoints:
        - "websecure"
      rule: "Host(`grafana.domain.xyz`)"
      middlewares:
        - default-headers
      service: grafana
    homeassistant:
      entryPoints:
        - "websecure"
      rule: "Host(`home.domain.xyz`)"
      middlewares:
        - default-headers
    influx:
      entryPoints:
        - "websecure"
      rule: "Host(`influxdb2.domain.xyz`)"
      middlewares:
        - default-headers
  services:
    grafana:
      loadBalancer:
        servers:
          - url: "http://192.168.1.223:3001"
    influx:
      loadBalancer:
        servers:
          - url: "http://192.168.1.223:8089"
    ha:
      loadBalancer:
        servers:
          - url: "https://192.168.1.201:8123"

If I copy the url directly it works. However, I cannot access via the domain.

On the logs I can see 404 errors from devices trying to access influxDB.

9

In every router you need to assign the according service.

1 Like
10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.