TLS Certificate subject mismatch in SSLLabs Test

Traefik Traefik v2
,
1

Hello,

I'm testing out Traefik with TLS 1.3 and troubleshooting the issue that why it doesn't obey the TLS options in config (I believe this is a bug but not 100% sure). In the process, I found out that SSLLab test report a different certificate than the one I obtained from LE staging. I believe this is some kind of configuration issue but I couldn't spot it. Can someone help? Thanks.

Host: https://whoami.apex.op.sparvojo.pw/
SSLLab Report: https://www.ssllabs.com/ssltest/analyze.html?d=whoami.apex.op.sparvojo.pw

docker-compose.yml

version: '3'

services:
  proxy:
    image: traefik:v2.0
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/etc/traefik/traefik.toml
      - ./conf.d/:/etc/traefik/conf.d/
      - ./acme:/acme
      - ./userfile.txt:/userfile.txt

  whoami:
    image: containous/whoami
    labels:
      - "traefik.http.middlewares.https-sts.headers.sslredirect=true"
      - "traefik.http.middlewares.https-sts.headers.stsSeconds=5259600"
      - "traefik.http.middlewares.https-headers.headers.browserXssFilter=true"
      - "traefik.http.middlewares.https-headers.headers.referrerPolicy=no-referrer-when-downgrade"
      - "traefik.http.middlewares.anti-ddos.ratelimit.average=5"
      - "traefik.http.middlewares.anti-ddos.ratelimit.burst=10"
      - "traefik.http.routers.whoami.rule=Host(`whoami.apex.op.sparvojo.pw`)"
      - "traefik.http.routers.whoami.entrypoints=web"
      - "traefik.http.routers.whoami.middlewares=https-sts"
      - "traefik.http.routers.whoami-https.rule=Host(`whoami.apex.op.sparvojo.pw`)"
      - "traefik.http.routers.whoami-https.entrypoints=web-secure"
      - "traefik.http.routers.whoami-https.middlewares=anti-ddos,https-headers,https-sts"
      - "traefik.http.routers.whoami-https.tls=true"
      - "traefik.http.routers.whoami-https.tls.options=highSecurity@file"
      - "traefik.http.routers.whoami-https.tls.certresolver=lestaging"

tls.toml (dynamic config via file provider)

[tls.options]
  [tls.options.highSecurity]
    minVersion = "VersionTLS13"
    cipherSuites = [
      "TLS_CHACHA20_POLY1305_SHA256"
    ]

  [tls.options.modern]
    minVersion = "VersionTLS12"
    cipherSuites = [
      "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
      "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
    ]

traefik.toml

[global]
  checkNewVersion = false
  sendAnonymousUsage = true

[api]
  dashboard = true

[entryPoints]
  [entryPoints.web]
    address = ":80"
  [entryPoints.web-secure]
    address = ":443"

[providers]
  [providers.file]
    directory = "/etc/traefik/conf.d"
    watch = true

  [providers.docker]
    endpoint = "unix:///var/run/docker.sock"

[certificatesResolvers.lestaging.acme]
  email = <stripped>
  storage = "acme/staging.json"
  caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
  [certificatesResolvers.lestaging.acme.httpChallenge]
    entrypoint = "web"

[certificatesResolvers.leprod.acme]
  email = <stripped>
  storage = "acme/production.json"
  [certificatesResolvers.leprod.acme.httpChallenge]
    entrypoint = "web"

[log]
  level = "DEBUG"
2

Inspect traefik log for clues.

3

I'm sorry for not including the logs and some analysis. However, it turns out the log didn't tell much either.

partial log:

time="2019-11-05T02:06:19Z" level=debug msg="http: TLS handshake error from 64.41.200.102:42720: tls: client offered only unsupported versions: [300]"
time="2019-11-05T02:06:19Z" level=debug msg="Serving default certificate for request: \"\""
time="2019-11-05T02:06:19Z" level=debug msg="http: TLS handshake error from 64.41.200.102:42820: EOF"
time="2019-11-05T02:06:19Z" level=debug msg="http: TLS handshake error from 64.41.200.102:42890: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:20Z" level=debug msg="http: TLS handshake error from 64.41.200.102:42978: tls: client offered only unsupported versions: [302 301 300]"
time="2019-11-05T02:06:20Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43074: tls: client offered only unsupported versions: [303 302 301 300]"
time="2019-11-05T02:06:20Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43172: tls: client offered only unsupported versions: [303 302 301 300]"
time="2019-11-05T02:06:21Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43286: read tcp 172.19.0.2:443->64.41.200.102:43286: read: connection reset by peer"
time="2019-11-05T02:06:21Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43436: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:22Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43502: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:22Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43596: tls: client offered only unsupported versions: [303 302 301 300]"
time="2019-11-05T02:06:22Z" level=debug msg="Serving default certificate for request: \"sni-test.ssllabs.com\""
time="2019-11-05T02:06:22Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43690: EOF"
time="2019-11-05T02:06:23Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43788: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:23Z" level=debug msg="Serving default certificate for request: \"\""
time="2019-11-05T02:06:23Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43866: EOF"
time="2019-11-05T02:06:23Z" level=debug msg="http: TLS handshake error from 64.41.200.102:43944: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:24Z" level=debug msg="http: TLS handshake error from 64.41.200.102:44026: tls: client offered only unsupported versions: [301 300]"
time="2019-11-05T02:06:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Connection\":[\"Close\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"64.41.200.102\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"64.41.200.102:44228\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2019-11-05T02:06:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://172.19.0.3:80" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Connection\":[\"Close\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"64.41.200.102\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"64.41.200.102:44228\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2019-11-05T02:06:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Connection\":[\"Close\"],\"User-Agent\":[\"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"64.41.200.102\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"64.41.200.102:44228\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2019-11-05T02:06:25Z" level=debug msg="http: TLS handshake error from 64.41.200.102:44476: read tcp 172.19.0.2:443->64.41.200.102:44476: read: connection reset by peer"
time="2019-11-05T02:06:26Z" level=debug msg="http: TLS handshake error from 64.41.200.102:44582: read tcp 172.19.0.2:443->64.41.200.102:44582: read: connection reset by peer"
time="2019-11-05T02:06:26Z" level=debug msg="http: TLS handshake error from 64.41.200.102:44700: read tcp 172.19.0.2:443->64.41.200.102:44700: read: connection reset by peer"

You can see it said Serving default certificate for request: "sni-test.ssllabs.com" in 2019-11-05T02:06:22Z but prior to that, is a bunch of TLS handshake error which SSLLabs tested for vulnerabilities. I'm avoiding to do TCP dump because 1) I really don't know how to do it with docker, and 2) I have no experiences in look at TCP packets to troubleshoot something. In 2019-11-05T02:06:25Z, it just serve the page without any problem. Here's a log where I access from my computer (I stripped my IP):

time="2019-11-05T02:01:54Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cache-Control\":[\"max-age=0\"],\"Dnt\":[\"1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"hidden\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"hidden:51677\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2019-11-05T02:01:54Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cache-Control\":[\"max-age=0\"],\"Dnt\":[\"1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"hidden\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"hidden:51677\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.19.0.4:80"
time="2019-11-05T02:01:54Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cache-Control\":[\"max-age=0\"],\"Dnt\":[\"1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0\"],\"X-Forwarded-Host\":[\"whoami.apex.op.sparvojo.pw\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"228c8f68aa88\"],\"X-Real-Ip\":[\"hidden\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"whoami.apex.op.sparvojo.pw\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"hidden:51677\",\"RequestURI\":\"/\",\"TLS\":null}"

By comparing the Request value, I can tell there isn't much difference. With all this, the only reason I can think of, is those intentional SSL error caused by the test somehow bugged Traefik. However, I'm not sure.

4

Okay. I think that you cannot rely on ssl labs giving correct results with a fake cert. Who knows what it does behind the scenes.

My guess would be that when it has problem validating the cert it tries accessing without SNI and then displays you the result.

There is nothing to worry about, I'd say.

1 Like
5

I just renew the certificate with the LE prod ACME server but SSLLabs still show the same result as subject mismatch.

Deleted

I personally believe this isn't Traefik fault but the testing method used by SSLLabs. In their website footnote:

4. Interoperability issues

In some rare cases trust cannot be established because of interoperability issues between our code and the code or configuration running on the server. We manually review such cases, but if you encounter such an issue please feel free to contact us. Such problems are very difficult to troubleshoot and you may be able to provide us with information that might help us determine the root cause.

I know it will be alright but still trigger me little bit.

I wounder if it is a bug. When ssl labs sents a sslv3 request, Traefik shouldn't be serving a certificate but terminate the connection (or whatever appropriate). I wasn't able to recreate in CURL but it actually what Traefik is doing.

6

I just checked with SSL Labs again. The wrong certificate is used when accessing without SNI, and proved @zespri is right.