Unable to use own certificate

Traefik Traefik v2 (latest)
1

Hello,

I'm trying to use a wildcard SSL certificate instead of traefik default cert. but I have seen below error logs in container. Please help on this..

Thanks

[root@manager ingress]# docker logs a75df542351a | grep error
time="2021-12-05T18:25:43Z" level=error msg="Error while creating certificate store: failed to load X509 key pair: tls: failed to find any PEM data in certificate input" tlsStoreName=default
time="2021-12-05T18:25:43Z" level=error msg="Unable to append certificate /root/ingress/certs/tls.crt to store: unable to generate TLS certificate : tls: failed to find any PEM data in certificate input" tlsStoreName=default
time="2021-12-05T18:25:43Z" level=error msg="Error during the build of the default TLS configuration: TLS store default not found" entryPointName=websecure
time="2021-12-05T18:25:43Z" level=error msg="Error during the build of the default TLS configuration: TLS store default not found" entryPointName=traefik
time="2021-12-05T18:25:43Z" level=error msg="Error while creating certificate store: failed to load X509 key pair: tls: failed to find any PEM data in certificate input" tlsStoreName=default
time="2021-12-05T18:25:43Z" level=error msg="Unable to append certificate /root/ingress/certs/tls.crt to store: unable to generate TLS certificate : tls: failed to find any PEM data in certificate input" tlsStoreName=default
time="2021-12-05T18:25:43Z" level=error msg="Error during the build of the default TLS configuration: TLS store default not found" entryPointName=websecure
time="2021-12-05T18:25:43Z" level=error msg="Error during the build of the default TLS configuration: TLS store default not found" entryPointName=traefik

docker-compose.yml

version: "3.9"

services:

  traefik:
    image: "traefik:latest"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--entrypoints.websecure.address=:443"
      - "--providers.file.directory=/certs/"
      - "--providers.file.watch=true"
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "/root/ingress/certs/:/certs/"
    networks:
      - traefik
    deploy:
      placement:
        constraints: [node.role == manager]

networks:
  traefik:

certs.yml

[root@manager ingress]# cat certs/certs.yml
tls:
  certificates:
    - certFile: "/root/ingress/certs/tls.crt"
      keyFile: "/root/ingress/certs/tls.key"
      stores:
        - default
  stores:
    default:
      defaultCertificate:
        certFile: "/root/ingress/certs/tls.crt"
        keyFile: "/root/ingress/certs/tls.key"

certs

[root@manager ingress]# ll certs/
total 12
-rw-r--r-- 1 root root  288 Dec  5 23:55 certs.yml
-rw-r--r-- 1 root root 2427 Dec  4 14:51 tls.crt
-rw-r--r-- 1 root root 1704 Dec  4 14:51 tls.key
2

It doesn't find the certs when the container is up.
Try to change the location like this:

tls:
  certificates:
    - certFile: "/certs/tls.crt"
      keyFile: "/certs/tls.key"
3

Thanks,

I already changed the same and now I can see our SSL certs. but I can see some errors in container logs like below..

[root@manager proxy]# docker logs 208fdb744389 | grep error
time="2021-12-06T07:35:32Z" level=debug msg="http: TLS handshake error from 10.0.0.2:52254: remote error: tls: unknown certificate"
time="2021-12-06T07:35:40Z" level=debug msg="http: TLS handshake error from 10.0.0.2:52256: remote error: tls: unknown certificate"
time="2021-12-06T08:46:37Z" level=debug msg="http: TLS handshake error from 10.0.0.2:52926: remote error: tls: unknown certificate"

4

Try to comment all lines in block stores in certs.yml:

tls:
  certificates:
    - certFile: /certs/tls.crt
      keyFile: /certs/tls.key
      #stores:
      #  - default
  #stores:
  #  default:
  #    defaultCertificate:
  #      certFile: "/root/ingress/certs/tls.crt"
  #      keyFile: "/root/ingress/certs/tls.key"

Diego

5

I think your static content is incomplete try to add ...

command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--entrypoints.websecure.address=:443"
      - "--providers.file.directory=/certs/"

      - "--providers.docker.exposedbydefault=true"
      - "--providers.docker.network=traefik"  # check your traefik network
      - "--providers.docker.swarmmode=true". # By the default is false

Check all options of docker provider --> CLI - Traefik

PD. I prefer to pass static and dynamic content in files instead cli (easier to see)

6

Thanks, I added the static content but still I see same unknow certificate error.. and also I tested host rule for ngnix service with tls labels and rule is working but it take more time to reach ngnix welcome page..

time="2021-12-06T18:47:02Z" level=debug msg="Serving default certificate for request: "nginx.example.com""
time="2021-12-06T18:47:02Z" level=debug msg="http: TLS handshake error from 10.0.0.2:51934: remote error: tls: unknown certificate"
time="2021-12-06T18:47:02Z" level=debug msg="Serving default certificate for request: "nginx.example.com""
time="2021-12-06T18:47:02Z" level=debug msg="http: TLS handshake error from 10.0.0.2:51935: remote error: tls: unknown certificate"
time="2021-12-06T18:47:48Z" level=debug msg="Serving default certificate for request: "nginx.example.com""
time="2021-12-06T18:47:48Z" level=debug msg="http: TLS handshake error from 10.0.0.2:51940: remote error: tls: unknown certificate"

7

I'm using my own certificates also in all my traefik services, so please double check your tls files (crt and key) are fine (no extra space or something).

If the tls files look fine It looks a networking issue when 'traefik' tries to resolve the key pair. Can you ping here how you created the traefik network?

Check your traefik network has attributes as: "driver--> overlay", "scope --> swarm".

8

Thanks and sorry for late reply..

certificate provided by Go Daddy I verified the crts is fine only and I find out the issue for why so much take time to reach the nginx welcome page.. that was my fault only. I saved so many DNS entries with same name different IP's. so it's trying to connect old entries..

For network part I created the same way only with overlay driver and the swarm scope.

So now I have only one problem is unknown certificate and bad certificate errors in container logs..

time="2021-12-11T12:46:59Z" level=debug msg="http: TLS handshake error from 10.0.0.2:57826: remote error: tls: unknown certificate"
time="2021-12-11T12:46:59Z" level=debug msg="http: TLS handshake error from 10.0.0.2:57827: remote error: tls: unknown certificate"
time="2021-12-11T12:50:29Z" level=debug msg="http: TLS handshake error from 10.0.0.2:57912: remote error: tls: bad certificate"
time="2021-12-11T12:50:29Z" level=debug msg="http: TLS handshake error from 10.0.0.2:57913: remote error: tls: bad certificate"

9

I am facing the same problem and trying to find out why?
Were you able to resolve this issue?. If yes, how?
TIA

10

Example to run multiple Traefik instances with custom TLS certs in Docker Swarm.

docker-compose.yml adds the TLS cert files as Docker Secret and config files as Docker Config, so they are available on all nodes. Docker networks created externally (manually on CLI).

# docker-compose.yml
version: '3.9'

services:
  traefik:
    image: traefik:v2.10.1
    command: --configFile=/traefik.yml
    hostname: '{{.Node.Hostname}}'
    ports:
      # listen on host ports without ingress network
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8080
        published: 8080
        protocol: tcp
        mode: host
      # port 8082 for metrics only used on internal network
    networks:
      - proxy
      - monitoring
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/log:/var/log
      #- traefik-certificates:/certificates
    configs:
      - traefik.yml
      - traefik-dynamic.yml
    secrets:
      - example.com.crt
      - example.com.key
    deploy:
      mode: global
      placement:
        constraints:
          - node.role==manager
      #resources:
      #  limits:
      #    cpus: '2'
      #    memory: 256M
      #  reservations:
      #    cpus: '1'
      #    memory: 256M
      update_config:
          parallelism: 1
          delay: 25s # longer than swarmModeRefreshSeconds
          order: stop-first # need to free ports first
          monitor: 5s
          failure_action: rollback
      labels:
        - prometheus-job=traefik
        - prometheus-port=8082

  whoami:
    image: traefik/whoami:v1.8
    hostname: '{{.Node.Hostname}}'
    networks:
      - proxy
    deploy:
      mode: global
      placement:
        constraints:
          - node.labels.type==app
      labels:
        - 'traefik.enable=true'
        #- 'traefik.http.routers.whoami.tls=true'
        - 'traefik.http.routers.whoami.entrypoints=websecure'
        - 'traefik.http.routers.whoami.rule=PathPrefix(`/whoami`)'
        - 'traefik.http.routers.whoami.priority=1024'
        - 'traefik.http.services.whoami.loadbalancer.server.port=80'
      resources:
        limits:
          cpus: '0.5'
          memory: 32M
        reservations:
          cpus: '0.1'
          memory: 16M

networks:
  proxy:
    name: proxy
    #driver: overlay
    #attachable: true
    external: true
  monitoring:
    name: monitoring
    #driver: overlay
    #attachable: true
    external: true

#volumes:
#  traefik-certificates:

configs:
  traefik.yml:
    file: ./traefik.yml
  traefik-dynamic.yml:
    file: ./traefik-dynamic.yml

secrets:
  example.com.crt:
    file: ./certs/2024/example.com.crt
  example.com.key:
    file: ./certs/2024/example.com.key

traefik.yml creates the providers and entrypoints. Note that certResolver does not work when having multiple Traefik instances. We run a managed LoadBalancer in front of the Traefik instances, that's why we use proxyProtocol.

# traefik.yml
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    swarmMode: true
    exposedByDefault: false
    network: proxy
    swarmModeRefreshSeconds: 10
    #httpClientTimeout: 10
  file:
    filename: /traefik-dynamic.yml
    watch: true
  #http:
  #  endpoint: "http://traefik_certbot/traefik-certbot.yml"
  #  pollInterval: 15s
  #  pollTimeout: 5s

entryPoints:
  web:
    address: :80
    proxyProtocol:
      trustedIPs:
        - 1.2.3.4/16
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          priority: 1000 # needed for certbot
  websecure:
    address: :443
    http:
      tls: true
    proxyProtocol:
      trustedIPs:
        - 1.2.3.4/16
  traefik:
    address: :8080
  metrics:
    address: :8082

api:
  debug: true
  dashboard: true
  insecure: false

metrics:
  prometheus:
    entryPoint: metrics
    addEntryPointsLabels: true
    addRoutersLabels: true
    addServicesLabels: true
    buckets:
      - 0.1
      - 0.3
      - 1.2
      - 5.0

log:
  level: INFO

accesslog:
  filepath: /var/log/traefik-access.log
  bufferingsize: 10
  format: json
  fields:
    defaultmode: keep
    headers:
      defaultmode: keep

#certificatesResolvers:
#  myresolver:
#    acme:
#      email: mail@example.com
#      storage: /certificates/acme.json
#      httpchallenge:
#        entrypoint: web

traefik-dynamic.yml creates dashboard router and loads TLS certs.

# traefik-dynamic.yml
http:
  routers:
    dashboard:
      entryPoints:
        - traefik # only port 8080
      rule: PathPrefix(`/api`) || PathPrefix(`/dashboard`) || PathPrefix(`/debug`)
      service: api@internal
      middlewares:
        - auth
  middlewares:
    auth:
      basicAuth:
        users:
          - "test:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/"

tls:
  options:
    default:
      minVersion: VersionTLS12
  certificates:
    - certFile: /run/secrets/example.com.crt
      keyFile: /run/secrets/example.com.key
  stores:
    default:
      defaultCertificate:
        certFile: /run/secrets/example.com.crt
        keyFile: /run/secrets/example.com.key

Additional note: Traefik LetsEncrypt is not clustered in the community edition. So we created a little certbot service to create the LE certs and then provide them via HTTP to the Traefik instances (link), that's why priority is in there. But this is not actively used.