Issues with TLS configuration in v2

Traefik Traefik v2
1

Hi, I've been trying to get TLS with existing certificates working in v2 with a fair amount of headache. After finally getting a https endpoint working, traefik v2 seems unable to marry provided certificates up with the services that need to use them - I simply can not see how this is achieved. In v1 this was very straight forward, not so much in v2.

For example, I have 1 self-signed for host.subdomain.domain.net, 1 commercial wildcard for *.domain.net, TLS providers file as follows:

[tls]
  [tls.options]
    [tls.options.default]
      minVersion = "VersionTLS12"
      sniStrict = false
  [[tls.certificate]]
    certFile = "/mysite/ssl/host.subdomain.domain.net.pem"
    keyFile = "/mysite/ssl/host.subdomain.domain.net.key"
  [[tls.certificate]]
    certFile = "/mysite/ssl/wildcard.domain.net.pem"
    keyFile = "/mysite/ssl/wildcard.domain.net.key"
  [[tls.certificate]]
    certFile = "/mysite/ssl/host.domain.net.pem"
    keyFile = "/mysite/ssl/host.domain.net.key"

example docker service labels:

    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.services.host.loadbalancer.server.port=80"
        - "traefik.http.routers.host.entryPoints=http"
        - "traefik.http.routers.host.rule=Host(`host.domain.net`)"
        - "traefik.http.routers.host.service=host"
        - "traefik.http.services.host-secure.loadbalancer.server.port=80"
        - "traefik.http.routers.host-secure.entryPoints=https"
        - "traefik.http.routers.host-secure.rule=Host(`host.domain.net`)"
        - "traefik.http.routers.host-secure.tls=true"
        - "traefik.http.routers.host-secure.tls.domains[0].main=host.domain.net"
        - "traefik.http.routers.host-secure.tls.domains[0].sans=host.domain.net"
        - "traefik.http.routers.host-secure.service=host-secure"

in this exmaple, wildcard.domain.net.pem and host.domain.net.pem are actually the same cert and key, as I was testing if traefik resolves the cert by filename or something since it was refusing to apply *.domain.net's cert to host.domain.net. I've tried with and without speficying ..tls.domains[0].main/sans. If I use tls.stores.default.defaultCertificate in the files provider I can specify a cert to use globally, and that works, however it's invalid to apply the wildcard to a subdomain hostname, that host needs to use its self-signed cert for now. If I remove that default cert, then traefik sends its own self-signed for all hosts whether they match a CN/SAN/filename of a defined cert or not.

What gives? How can I tell traefik v2 to use THIS cert for THIS hostname and THAT cert for THAT hostname, if it (apparently) isn't matching on CN, SAN, or knows that a wildcard it's been given is applicable?

2

I've managed to turn up logging to DEBUG, and this is what traefik2 is reporting with strict sni enabled:

http: TLS handshake error from xxx.xxx.xxx.xxx:55770: strict SNI enabled - No certificate found for domain: "host.domain.net", closing connection"

This is despite quadruple-checking the cert is valid, has host.domain.net as CN and SAN, has the right key, both readable by traefik2, and the dynamic ssl toml file following the format from the documentation exactly.

How does this ever work? Has anyone ever actually managed to get Traefik v2 working with provided certs at all and not just acme?

Can anyone give me a working example of service labels + traefik config toml + ssl dynamic config toml ? Please?

3

These options are only for Let's Encrypt.

Your certificates must contains SNI information.

1 Like
4

I've tried with and without the tls.domains[] label. My certificates certainly do contain the right information.

Can you confirm the format of my ssl.toml is correct? It's as if Traefik isn't seeing the certificate stanzas at all - or if it is then it is completely failing to parse CN and SAN within the certificates, but the debug logs make no mention of this beyond claiming no certificate exists for [hostname], which is incorrect

5

Have you mount your certificates?

Could you provide your complete configuration (docker-compose file, ...)

6

stack.yml:

version: '3.7'

services:
  traefik2:
    image: traefik:2.0
    command: --configFile=/domain/traefik.toml
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    networks:
      - proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/containers/traefik2:/domain
    deploy:
      mode: global

networks:
  proxy:
    driver: overlay
    attachable: true
    name: proxy
    ipam:
      driver: default
      config:
        - subnet: 10.10.0.0/24

/mnt/containers/traefik2/traefik.toml:

[global]
  sendAnonymousUsage = false
  checkNewVersion = true

[log]
  level = "DEBUG"
  filePath = "/domain/traefik.log"

[accessLog]
  filePath = "/domain/access.log"

[api]
  insecure = true
  dashboard = true

[providers]
  [providers.docker]
    endpoint = "unix:///var/run/docker.sock"
    watch = true
    swarmMode = true
    network = "proxy"
    exposedByDefault = false
  [providers.file]
    filename = "/domain/ssl/ssl.toml"
    watch = true

[entryPoints]
  [entryPoints.http]
    address = ":80"
  [entryPoints.https]
    address = ":443"

/mnt/containers/traefik2/ssl/ssl.toml:

[tls]
  [tls.options]
    [tls.options.default]
      minVersion = "VersionTLS12"
      sniStrict = true

  [[tls.certificate]]
    certFile = "/domain/ssl/host.domain.crt"
    keyFile = "/domain/ssl/host.domain.key"

(also tried with sniStrict = false, with and without the top level [tls] header, [[tls.certificate]] inside the [tls] block, outside it)

other service stack.yml:

  host:
    image: registry.domain/host:latest
    command: ["npm","start"]
    networks:
      - proxy
    volumes:
      - "/mnt/containers/host:/data"
    deploy:
      replicas: 1
      labels:
        - "traefik.enable=true"
        - "traefik.http.services.host-secure.loadbalancer.server.port=3000"
        - "traefik.http.routers.host-secure.tls=true"
        - "traefik.http.routers.host-secure.entryPoints=https"
        - "traefik.http.routers.host-secure.rule=Host(`host.domain`)"
        - "traefik.http.routers.host-secure.service=host-secure"

Tried many, many combinations of labels here. this is the current 'rip everything up and start with basic functionality' format I've arrived at - traefik still isn't serving the certificate for it. Additionally, it's responding to the client in plain http instead of https despite .tls=true, causing the client to throw an ssl protocol error now.

7

Already figured out. You are missing the file provider...

8

But I did define the file provider for the tls options and cert definitions


  [providers.file]
    filename = "/domain/ssl/ssl.toml"
1 Like
9

Uh, sorry. Didn‘t see that.
Have you tried exec sh into the running container and check if the mounted files are readable?

10

Yep, they are present and readable. I know it shouldn't make a difference but this weekend I'll try specifying the config in yaml instead of toml in case there's weirdness in the parser or my formatting. I'm not really sure what more to try beyond that.

11

I've given up for now and have opted for a different solution as this was taking up way more time than it should. To anyone else ending up here trying to find a solution to a similar problem with premade tls certs - sorry, I don't have an answer for you.

12

I decided to revisit this, being a slow day at work on xmas eve, and managed to get it working. I also see why it was failing for me previously so thought I'd report back here for future users experiencing the same issue.

The problem was I had [[tls.certificate]] in my tls config toml instead of [[tls.certificates]]. I see the documentation was changed about 6 months ago from the former to the latter, and perhaps when I decided to revisit traefik2 after it went stable I was still using an old ssl.toml from prior experimentation.

It would have saved me a lot of headache though if traefik had reported 'unrecognized configuration item' or something instead of silently failing over that section and appearing to accept the config.

13

(<opinion>side note: TOML's a lot harder to read than it admits... I don't see the benefit of it tbh.
I think it says a bit that the other people in this thread also easily missed that tiny typo amongst toml's thick forrest of square brackets :stuck_out_tongue:</opinion>)