Traefik Gateway API Not Serving ACME Certificate and Showing Default Certificate Errors

[Tockra]

Hello everyone,

I’m currently trying to set up Traefik in my Kubernetes cluster using the official Helm chart. So far, the Traefik pods are reachable and respond with a 404, which is expected at this stage. However, I tried adding an application and assumed Traefik would automatically create a route for it. This does not seem to happen, and I suspect something might be wrong with the Gateway generated by the Helm chart.

Here is my installation command:

kubectl create namespace traefik
helm upgrade --install \
  traefik traefik/traefik \
  --namespace traefik \
  --values traefik-values.yaml

Below is my traefik-values.yaml:

# traefik-values.yaml

deployment:
  replicas: 2
service:
  externalIPs:
    - "<MY_PUBLIC_FLOATING_IP>"  # <-- placeholder for your public IP

# Let's Encrypt (ACME) configuration
certificatesResolvers:
  default:
    acme:
      email: "dev@mydomain.de"       # <-- Use your valid email
      storage: "/data/acme.json"     # <-- Where to store certificate data in the pod
      httpChallenge:
        entryPoint: "web"            # Must be served on port 80

# Ports configuration: set up both HTTP (80) and HTTPS (443)
ports:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: ":443"
    http:
      tls:
        certResolver: default

# Use a PVC to persist /data
persistence:
  enabled: true
  name: data
  storageClass: "longhorn"   # <-- Example RWX storage class
  accessMode: ReadWriteMany
  size: 1Gi

# Basic logging to see what's happening
logs:
  general:
    level: INFO
  access:
    enabled: true

providers:
  # Disable the Ingress provider
  kubernetesIngress:
    enabled: false
  # Enable the Gateway API provider
  kubernetesGateway:
    enabled: true

gateway:
  enabled: true
  namespacePolicy: All
  entryPoints:
    - web
    - websecure

Gateway Description (kubectl describe gateway -n traefik traefik-gateway):

Name:         traefik-gateway
Namespace:    traefik
...
Spec:
  Gateway Class Name:  traefik
  Listeners:
    Name:      web
    Port:      8000
    Protocol:  HTTP
    AllowedRoutes:
      Namespaces:
        From:  Same
...

HTTPRoute:

Name:         gitea
Namespace:    web-applications
...
Spec:
  Hostnames:
    git.mydomain.de
  ParentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: traefik-gateway
      namespace: traefik
      sectionName: websecure
  Rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - kind: Service
          name: gitea
          port: 3000
          weight: 1
...

When I visit git.mydomain.de, I see an “unsafe certificate” warning, and the logs show:

<DATE/TIME> DBG ... > Serving default certificate for request: "git.mydomain.de"
<DATE/TIME> DBG ... > http: TLS handshake error from <IP>: remote error: tls: unknown certificate

Has anyone run into a similar issue or have any ideas on what might be missing in my configuration? I suspect there’s a mismatch between the sectionName: websecure in the HTTPRoute and the actual listener in the Gateway, but I’m not entirely sure. Any pointers would be greatly appreciated!

T

[bluepuma77]

Usually cert-manager is used for TLS with Traefik in Kubernetes (guide).

[Tockra]

In both cases (Ingress and Gateway API?).
It feels a little bit odd to add the certRef for every certifcate into the gateway by myself.
So I have to create a HTTPRoute + A Certificate Resource + Change the Gateway Resource
everytime when I want to add a new application.
In my traefik docker setup, I just have to set the labels of my new application and everything works fine out of the box.

[bluepuma77]

If have no experience with Kubernetes.

The cert-manager is usually used because Traefik itself (certResolver) does not handle clustered/distributed LetsEncrypt (running multiple instances of Traefik), except for the Traefik enterprise version.

I actually wonder that httpChallenge works for you with 2 Traefik replicas. The standard issue is that one instance will trigger external LE verify, that external request ends on the other instance which does not know the validation token, then no TLS cert is issued.

[Tockra]

Okay.
I'm using cert-manager now, which handles CRDs called "Certificates". You create a new certificate resource which describes your domain and ends in signed TLS cert, which is stored in the kubernetes secret space.
Here you have two tricks:

1. You create a certificate containing all domains you which to handle with your traefik instance e.g. sub1.mydomain.com, sub2.mydomain.com, sub1.myotherdomain.com, ... .
   Then you can add this secret in your traefik helm chart values.yaml, which will create a gateway for you, listing on 80 and 443 which contains your secret.
   The big problem here is, that you can only define ONE kubernetes secret with tls cert. So you need to add all your domains to that one certificate.
2. Another (better) solution is to use the DNS challenge method of lets encrypt (if you can). This way your are able to create certificates with wildcard sub domains e.g. *.mydomain.com . This way a person visiting your sub1 webpage cannot find sub2.mydomain.com by checking the certificate.

The second case is the better way in my opinion, because to add a new application with a new subdomain you just have to add a "httproute" resource which explains your traefik where to route the request.
The first solution means, that you have to edit your existing "certificate" CRD and add your now subdomain there. Otherwise the routing would work, but your connection will be encrypted with a traefik default cert, which is not trusted by your browser by default.