Problem: Traefik Include Domains from external file and route to container

Traefik Traefik v3 (beta)
,
1

Hey Guys :smile:
I hope you had a great christmas and a good start in the new year.

I have a small Problem with my traefik configuration. I am very new to traefik and in the past i only used a simple docker-compose.yml configuration to include domains for my service. The problem is that the domains to be added a variable, because users can add domains for the service in the dashboard panel. I coded a script which is adding the domains to a yml (tryed txt as well) file and docker should be able to read the files and route them to the store container.

I tried different configurations (with txt and yml) but it does not work.
At the moment the domain gets the SSL certificate, but is not routed to the service. (Maybe the SSL certificate is still active from the past configuration.
There is no error message when starting with docker compose up.

It would be really nice if you can give me an idea how to solve that issue. I wasted 8+ hours of my lifetime already haha :smile_cat:

/data/docker-compose.yml

version: "3"
services:
  traefik:
    image: traefik:latest
    restart: unless-stopped
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /data/acme.json:/acme.json
      - /data/traefik.yaml:/traefik.yaml:ro
      - /data/traefik/traefik:/traefik
    networks:
      - kt-proxy
  store:
    image: store:latest
    restart: unless-stopped
    labels:
      - "traefik.http.routers.store-router.tls=true"
      - "traefik.http.routers.store-router.entrypoints=web-secure"
      - "traefik.http.routers.store-router.tls.certresolver=myresolver"
    depends_on:
      - traefik
    networks:
      - kt-proxy
networks:
  kt-proxy:
    name: kt-proxy
    external: true

/data/traefik.yaml

providers:
  file:
    directory: "/traefik"
    filename: "domains.yml"
    watch: true

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: "web-secure"
          scheme: "https"
  web-secure:
    address: ":443"

certificatesResolvers:
  myresolver:
    acme:
      email: "developer@domain.tdl"
      storage: "acme.json"
      httpChallenge:
        entryPoint: "web"

services:
  store:
    labels:
      - "traefik.http.routers.store-router.rule=Host(`.+`)"
      - "traefik.http.routers.store-router.tls=true"
      - "traefik.http.routers.store-router.entrypoints=web-secure"
      - "traefik.http.routers.store-router.tls.certresolver=myresolver"


#http:
#  routers:
#    store-router: {}

/data/traefik/traefik/domains.yml

http:
  routers:
    store-router:
      entryPoints:
        - web-secure
      service: store
      rule: "Host(`subdomain.subdomain.domain1.com`,`domain2.com`,`domain3.com`)"

Thanks heaps :smile:

Greets,
MoBeMo

2

Debug Log

[+] Running 2/0
 â ¿ Container data-traefik-1        Created                                                                                                                                                             0.0s
 â ¿ Container data-store-1  Created                                                                                                                                                             0.0s
Attaching to data-store-1, data-traefik-1
canceled
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Configuration loaded from file: /traefik.yaml"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Traefik version 2.9.5 built on 2022-11-17T15:04:26Z"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"web-secure\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"web-secure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"file\":{\"directory\":\"/traefik\",\"watch\":true,\"filename\":\"domains.yml\"}},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"myresolver\":{\"acme\":{\"email\":\"developer@domain.tld\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"web\"}}}}}"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Starting TCP Server" entryPointName=web
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Starting TCP Server" entryPointName=web-secure
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Starting provider *file.Provider"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="*file.Provider provider configuration: {\"directory\":\"/traefik\",\"watch\":true,\"filename\":\"domains.yml\"}"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Starting provider *traefik.Provider"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="*traefik.Provider provider configuration: {}"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Starting provider *acme.Provider"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="*acme.Provider provider configuration: {\"email\":\"developer@domain.tld\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"web\"},\"ResolverName\":\"myresolver\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" providerName=myresolver.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=info msg="Testing certificate renew..." providerName=myresolver.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"store-router\":{\"entryPoints\":[\"web-secure\"],\"service\":\"store\",\"rule\":\"Host(`test.store.dev.domain.tld`)\"}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=file
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"acme-http\":{\"entryPoints\":[\"web\"],\"service\":\"acme-http@internal\",\"rule\":\"PathPrefix(`/.well-known/acme-challenge/`)\",\"priority\":2147483647},\"web-to-web-secure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-web-secure\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"acme-http\":{},\"noop\":{}},\"middlewares\":{\"redirect-web-to-web-secure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=myresolver.acme
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test.store.dev.domain.tld,vegan.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) 0711rave.store.dev.domain.tld,test.store.dev.domain.tld,vegan.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test44.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test110.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test.store.dev.domain.tld,test100.store.dev.domain.tld,test101.store.dev.domain.tld,test102.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:57Z" level=debug msg="Adding certificate for domain(s) test300.store.dev.domain.tld"
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Added outgoing tracing middleware acme-http@internal" middlewareType=TracingForwarder routerName=acme-http@internal entryPointName=web middlewareName=tracing
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Added outgoing tracing middleware noop@internal" routerName=web-to-web-secure@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=web
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Creating middleware" middlewareName=redirect-web-to-web-secure@internal middlewareType=RedirectScheme entryPointName=web routerName=web-to-web-secure@internal
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Setting up redirection to https 443" entryPointName=web routerName=web-to-web-secure@internal middlewareName=redirect-web-to-web-secure@internal middlewareType=RedirectScheme
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Adding tracing to middleware" entryPointName=web routerName=web-to-web-secure@internal middlewareName=redirect-web-to-web-secure@internal
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery
data-traefik-1        | time="2023-01-09T10:10:58Z" level=error msg="the service \"store@file\" does not exist" routerName=store-router@file entryPointName=web-secure
data-traefik-1        | time="2023-01-09T10:10:58Z" level=debug msg="Creating middleware" entryPointName=web-secure middlewareName=traefik-internal-recovery middlewareType=Recovery
3

As far as I know Host() will only use a single domain, you can use Host() || Host() to add multiple.

You labels on the store seem redundant to the dynamic config file with domains. I don’t think they are even picked up because I don’t see a provider.docker for automatically reading them.

4

Hello the problem is i need to use regex because I want all hosts that are given in the yml file.

Do you know anything I could do to include the domains from the domain file and route them to the container?

5

I just checked the Traefik docs router rule:

Host(`example.com`, ...)` 
Check if the request domain (host header value) targets one of the given `domains`.

So you can use multiple domain names within Host. There is also HostRegexp.

It's not forwarding to your service because your service store is unknown to Traefik. Either you add provider.docker to let the Docker service be recognized including the used labels (you used store-router here) - or - you set up Traefik service manually with loadbalancer in your dynamic config with the domains.