DNS Challenge not working on kubernetes

Traefik Traefik v3 (latest)
,
1

I have a traefik DaemonSet on my kubernetes cluster, and I need to migrate from tls challenge to DNS.

We use route 53 on AWS for DNS and traefik on docker on our other machines works fine with this, but on kubernetes the DNS challenge doesn’t work and Traefik doesn’t log any errors.

Here is my values.yaml file for traefik (v 3.5.3), chart traefik-37.1.2

deployment:
  kind: DaemonSet
ingressClass:
  enabled: true
  isDefaultClass: false
  name: traefik-external
ports:
  hazelcast:
    expose:
      default: true
    exposedPort: 5701
    port: 5701
    protocol: TCP
providers:
  kubernetesIngress:
    ingressClass: traefik-external
service:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-name: eks-test1-external
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-type: external
  type: LoadBalancer

certificatesResolvers:
  myresolver:
    acme:
      email: "xxxxxxxxx"
      storage: "/data/acme.json"       # Path to store the certificate information.
      dnsChallenge:
        provider: route53

env: 
  - name: AWS_REGION
    value: xxxxxxxxx
  - name: AWS_HOSTED_ZONE_ID
    value: xxxxxxxxx
  - name: AWS_ACCESS_KEY_ID
    value: xxxxxxxxx
  - name: AWS_SECRET_ACCESS_KEY
    value: xxxxxxxxx

The logs only show this:

INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=myresolver.acme

But no errors, and my services get self signed certs.

2

I don’t use k8s, but I think usually cert-manager is used for distributed TLS in k8s (doc).

3

I’m sorry but I think I’m not understanding. I already have a working TLS configuration for traefik on k8s, but we want to phase that out in favor of DNS challenge.

The configuration is the same that we use on our non-k8s instances of traefik, which use docker compose with the following command:

    command:
      - "--log.level=WARN"

      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.network=traefik"

      - "--entryPoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"

      - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=route53"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.delayBeforeCheck=60"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53"
      - "--certificatesresolvers.myresolver.acme.email=xxxxxxxxxx"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"

The same configuration, which I posted in the first post, doesn’t seem to be working on k8s. I can’t understand what I’m doing wrong…

4

Enable Traefik DEBUG log (doc) and check for ERR, acme, tls.

Usually cert-manager is used in k8s, especially when you have multiple nodes/instances. Traefik LetsEncrypt is not cluster-enabled, you will run into issues with multiple instances and when instances are moved/replaced. LetsEncrypt has very strict creation limits and you may end up without a valid TLS cert.

5

You’re right, using cert-manager was the solution on k8s. Thanks for the help!