Self-signed certificates: failed to find any PEM data in certificate

Traefik Traefik v2
1

Hey there,

I have some issues getting traefik to work.

Related logs:

time="2022-12-30T17:21:02Z" level=debug msg="No store is defined to add the certificate certs/wiki.local.cert, it will be added to the default store."

time="2022-12-30T17:21:02Z" level=error msg="Unable to append certificate certs/wiki.local.cert to store: unable to generate TLS certificate : tls: failed to find any PEM data in certificate input" tlsStoreName=default

When I try to access wiki.local, it returns that it's unsafe, uses the default traefik certificate and returns a "404 page not found".

Docker-compose:

version: '3'
services:

  socket-proxy:
    container_name: socket-proxy
    image: tecnativa/docker-socket-proxy
    privileged: true
    environment:
      CONTAINERS: 1
      POST: 0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - socket_net
    restart: unless-stopped

  traefik:
    container_name: traefik
    image: traefik:v2.6
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.local`)"
      - "traefik.http.routers.traefik.entrypoints=web"
      - "traefik.http.routers.traefik.service=api@internal" 
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    volumes:
      - traefik:/etc/traefik/
    networks:
      - global_proxy
      - socket_net
      
  whoami:
    image: "traefik/whoami"
    container_name: "simple-service"
    networks:
      - global_proxy
    labels:
      traefik.enable: true
      traefik.http.routers.wiki.rule: Host(`wiki.local`) # domain to expose on
      traefik.http.routers.wiki.entrypoints: web" # if you named your 443 entrypoint differently than webscure, substitute it here!
      traefik.port: 80
      
volumes:
  traefik:
networks:
  socket_net:
  global_proxy:
    ipam:
      driver: default

traefik.yaml:

api:
  dashboard: true                             # Enable the dashboard
  insecure: true

entryPoints:
  web:
    address: ":80"                            # Create the HTTP entrypoint on port 80
    http:
      redirections:                           # HTTPS redirection (80 to 443)
        entryPoint:
          to: "websecure"                         # The target element
          scheme: "https"                     # The redirection target scheme
  websecure:
    address: ":443"                           # Create the HTTPS entrypoint on port 443

global:
  checknewversion: true                       # Periodically check if a new version has been released.
  sendanonymoususage: true                    # Periodically send anonymous usage statistics.

providers:
  docker:
    endpoint: "tcp://socket-proxy:2375"
    exposedByDefault: false                   # Only expose container that are explicitly enabled (using label traefik.enabled)
    network: "traefik_global_proxy"                    # Default network to use for connections to all containers.
    defaultRule: 'Host(`{{ normalize .Name | replace "-pi" "" }}.local`)'
    watch: true                               # Watch Docker Swarm events
  file:
    filename: "/etc/traefik/config.yaml"       # Link to the dynamic configuration
    watch: true                               # Watch for modifications
  providersThrottleDuration: 10               # Configuration reload frequency

serversTransport:
  insecureSkipVerify: true

log:
  level: DEBUG

config.yaml:

tls:
  certificates:
    - certFile: certs/crater.local.cert
      keyFile: certs/crater.local.key
    - certFile: certs/paperless.local.cert
      keyFile: certs/paperless.local.key
    - certFile: certs/wiki.local.cert
      keyFile: certs/wiki.local.key

openssl x509 -in certs/wiki.local.cert -text :

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 5 (0x5)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = DE, ST = RLP, L = Bingen, O = Name, emailAddress = info@.., CN = internal-ca
        Validity
            Not Before: Dec 30 17:07:29 2022 GMT
            Not After : Jan 31 17:07:29 2024 GMT
        Subject: C = DE, ST = RLP, L = Bingen, O = Name, emailAddress = info@..., CN = wiki.local
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    [...]
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Cert Type:
                SSL Server
            Netscape Comment:
                OPNsense Generated Server Certificate
            X509v3 Subject Key Identifier:
                9F:AA:D4:57:33:10:E4:BB:10:FF:57:45:47:69:3F:8D:1A:20:38:A6
            X509v3 Authority Key Identifier:
                keyid:B1:65:CC:3B:DB:9D:4B:7A:D9:E2:FF:FA:13:8D:1A:F5:F6:06:C1:75
                DirName:/C=DE/ST=RLP/L=Bingen/O=Name/emailAddress=info@.../CN=internal-ca
                serial:00
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, 1.3.6.1.5.5.8.2.2
            X509v3 Key Usage:
                Digital Signature, Key Encipherment
            X509v3 Subject Alternative Name:
                DNS:wiki.local
    Signature Algorithm: sha256WithRSAEncryption

These certificates have been generated by my local CA. If I generate the certificates manually using openssl, I have the same issue.

Do you have any ideas?

Thank you and best regards

2

I recommend to use an absolute full path (starting with /) and not a relative one.

Go into the Traefik container and check that the cert is readable.

Update your Traefik version to latest, probably v2.9.6.

3

Hi, thank you for your hint! After changing the path, the certificates were detected and seem to work now. Furthermore, I had a typo in my routes (http" instead of http).
So, I got no more errors or warnings in my logs, but still receive the 404 - do you have any more ideas?
The apache log of my webservices is empty - so it doesn't make it to them.

time="2022-12-30T21:28:45Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yaml"

time="2022-12-30T21:28:45Z" level=info msg="Traefik version 2.6.7 built on 2022-05-24T14:19:52Z"

time="2022-12-30T21:28:45Z" level=info msg="Stats collection is enabled."

time="2022-12-30T21:28:45Z" level=info msg="Many thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration."

time="2022-12-30T21:28:45Z" level=info msg="Help us improve Traefik by leaving this feature on :)"

time="2022-12-30T21:28:45Z" level=info msg="More details on: https://doc.traefik.io/traefik/contributing/data-collection/"

time="2022-12-30T21:28:45Z" level=info msg="Starting provider aggregator.ProviderAggregator"

time="2022-12-30T21:28:45Z" level=debug msg="Start TCP Server" entryPointName=web

time="2022-12-30T21:28:45Z" level=debug msg="Start TCP Server" entryPointName=traefik

time="2022-12-30T21:28:45Z" level=info msg="Starting provider *file.Provider"

time="2022-12-30T21:28:45Z" level=debug msg="*file.Provider provider configuration: {\"watch\":true,\"filename\":\"/etc/traefik/config.yaml\"}"

time="2022-12-30T21:28:45Z" level=debug msg="Start TCP Server" entryPointName=websecure

time="2022-12-30T21:28:45Z" level=info msg="Starting provider *traefik.Provider"

time="2022-12-30T21:28:45Z" level=debug msg="*traefik.Provider provider configuration: {}"

time="2022-12-30T21:28:45Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"

time="2022-12-30T21:28:45Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {\"Timeout\":20000000000}"

time="2022-12-30T21:28:45Z" level=info msg="Starting provider *docker.Provider"

time="2022-12-30T21:28:45Z" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"tcp://socket-proxy:2375\",\"defaultRule\":\"Host(`{{ normalize .Name | replace \\\"-pi\\\" \\\"\\\" }}.local`)\",\"network\":\"traefik_global_proxy\",\"swarmModeRefreshSeconds\":\"15s\"}"

time="2022-12-30T21:28:45Z" level=debug msg="Configuration received from provider file: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=file

time="2022-12-30T21:28:45Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect@internal\",\"dashboard_stripprefix@internal\"],\"service\":\"dashboard@internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645},\"web-to-websecure\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-web-to-websecure\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}},\"redirect-web-to-websecure\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"serversTransports\":{\"default\":{\"insecureSkipVerify\":true,\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"tls\":{}}" providerName=internal

time="2022-12-30T21:28:45Z" level=debug msg="No default certificate, generating one" tlsStoreName=default

time="2022-12-30T21:28:45Z" level=debug msg="Provider connection established with docker 20.10.20 (API 1.41)" providerName=docker

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=socket-proxy-traefik-2cbbf1b48a453b36604021bb592cfbc4c6620e815d325dadab6fbd2895d215b7

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=docker-autoheal-service-f3058561486ae34b1ba9b21bf58a774c8b0f7bd8da51c3e95d35e047452ad103

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" container=docker-portainer-agent-service-07dedf1e8b556f7a0c9d8a9990a864988177c3910fe1bd1a66c6deb70a8408ea providerName=docker

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=docker-portainer-service-e8d7df38f206eae7a9c86f3974541eb21594722cbbed6626731b7317f3df9058

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=cron-crater-1632292e9ec32839a8a150909c92c3d0348c4f41ec1452e5d5c9a2b9307f05fe

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=app-crater-b89bc996d507c6937945c7694e4fc5459ae3b9d17ff1ba6fae2d919e36661e05

time="2022-12-30T21:28:45Z" level=debug msg="Filtering disabled container" providerName=docker container=db-crater-9ae8189fa55eebce70acfd64704ccfcc19b51753c87c9e23cc55c4c6c13559df

time="2022-12-30T21:28:45Z" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"nginx\":{\"entryPoints\":[\"web\"],\"service\":\"nginx-crater\",\"rule\":\"Host(`crater.local`)\",\"tls\":{}},\"traefik\":{\"entryPoints\":[\"web\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.local`)\"},\"wiki\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami-traefik\",\"rule\":\"Host(`wiki.local`)\"}},\"services\":{\"nginx-crater\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://192.168.176.3:80\"}],\"passHostHeader\":true}},\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://192.168.176.4:80\"}],\"passHostHeader\":true}},\"whoami-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://192.168.176.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker

time="2022-12-30T21:28:45Z" level=debug msg="No store is defined to add the certificate MIIGxjCCBK6gAwIBAgIBAzANBgkqhkiG9w0BAQsFADB1MQswCQ, it will be added to the default store."

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) crater.local"

time="2022-12-30T21:28:45Z" level=debug msg="No store is defined to add the certificate MIIGzDCCBLSgAwIBAgIBBDANBgkqhkiG9w0BAQsFADB1MQswCQ, it will be added to the default store."

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) paperless.local"

time="2022-12-30T21:28:45Z" level=debug msg="No store is defined to add the certificate MIIGwjCCBKqgAwIBAgIBBTANBgkqhkiG9w0BAQsFADB1MQswCQ, it will be added to the default store."

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) wiki.local"

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing entryPointName=traefik routerName=api@internal middlewareType=TracingForwarder

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal middlewareName=tracing

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex

time="2022-12-30T21:28:45Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_redirect@internal entryPointName=traefik

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware noop@internal" entryPointName=web routerName=web-to-websecure@internal middlewareName=tracing middlewareType=TracingForwarder

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=web routerName=web-to-websecure@internal middlewareType=RedirectScheme middlewareName=redirect-web-to-websecure@internal

time="2022-12-30T21:28:45Z" level=debug msg="Setting up redirection to https 443" middlewareType=RedirectScheme middlewareName=redirect-web-to-websecure@internal entryPointName=web routerName=web-to-websecure@internal

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" middlewareName=redirect-web-to-websecure@internal entryPointName=web routerName=web-to-websecure@internal

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2022-12-30T21:28:45Z" level=debug msg="No default certificate, generating one" tlsStoreName=default

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) crater.local"

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) paperless.local"

time="2022-12-30T21:28:45Z" level=debug msg="Adding certificate for domain(s) wiki.local"

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware noop@internal" entryPointName=web routerName=web-to-websecure@internal middlewareName=tracing middlewareType=TracingForwarder

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=web routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme

time="2022-12-30T21:28:45Z" level=debug msg="Setting up redirection to https 443" routerName=web-to-websecure@internal middlewareName=redirect-web-to-websecure@internal middlewareType=RedirectScheme entryPointName=web

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" middlewareName=redirect-web-to-websecure@internal entryPointName=web routerName=web-to-websecure@internal

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder entryPointName=web routerName=traefik@docker middlewareName=tracing

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=websecure routerName=wiki@docker serviceName=whoami-traefik middlewareName=pipelining middlewareType=Pipelining

time="2022-12-30T21:28:45Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=wiki@docker serviceName=whoami-traefik

time="2022-12-30T21:28:45Z" level=debug msg="Creating server 0 http://192.168.176.2:80" routerName=wiki@docker serviceName=whoami-traefik serverName=0 entryPointName=websecure

time="2022-12-30T21:28:45Z" level=debug msg="child http://192.168.176.2:80 now UP"

time="2022-12-30T21:28:45Z" level=debug msg="Propagating new UP status"

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware whoami-traefik" entryPointName=websecure routerName=wiki@docker middlewareName=tracing middlewareType=TracingForwarder

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=websecure middlewareName=traefik-internal-recovery

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=api@internal middlewareName=tracing

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal middlewareName=tracing

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal entryPointName=traefik

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" middlewareType=RedirectRegex entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal

time="2022-12-30T21:28:45Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik

time="2022-12-30T21:28:45Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" middlewareName=pipelining middlewareType=Pipelining entryPointName=web routerName=nginx@docker serviceName=nginx-crater

time="2022-12-30T21:28:45Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=nginx@docker serviceName=nginx-crater

time="2022-12-30T21:28:45Z" level=debug msg="Creating server 0 http://192.168.176.3:80" routerName=nginx@docker serverName=0 serviceName=nginx-crater entryPointName=web

time="2022-12-30T21:28:45Z" level=debug msg="child http://192.168.176.3:80 now UP"

time="2022-12-30T21:28:45Z" level=debug msg="Propagating new UP status"

time="2022-12-30T21:28:45Z" level=debug msg="Added outgoing tracing middleware nginx-crater" routerName=nginx@docker middlewareName=tracing middlewareType=TracingForwarder entryPointName=web

time="2022-12-30T21:28:45Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2022-12-30T21:28:45Z" level=debug msg="Adding route for crater.local with TLS options default" entryPointName=web

Here, the log attached - but I cannot find any hint in there :frowning:

4

Grep your log for „error“, enable and check access log.

5

Maybe your labels are not recognized, normally used with =

labels:
      - "traefik.enable=true"