Traefik keeps showing TLS handshake error tls: unknown certificate

Traefik Traefik v3 (latest)
1

I've read numerous forum posts about TLS and Certificates and decided to get help specific to my case.

My environment follows

  1. A Domain Controller with DNS
  2. A second Windows Server which is joined to the Domain and has a DNS role too.
  3. Binary file of exe and configuration
  4. The 2 servers are VMs and a Domain User that is also a VM connected to each other within VLAN for testing

I configured the Domain Controller DNS to have a Conditonal Forwarding to the DNS to the second Windows Server. DNS resolves properly to the IP address of the second server.

I first tried the traefik dashboard, http works but I want it to be usable on https and route all http to https automatically. The following are my configurations.

traefik.yml

global:
  checkNewVersion: false
  sendAnonymousUsage: false
serverTransport:
  insecureSkipVerify: true
log:
  level: DEBUG
entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https
  https:
    address: ":443"
    http:
      tls: true
providers: 
  file:
    directory: dynamic/
    watch: true
api:
  dashboard: true

dynamic/traefik.yml

https:
  routers:
    traefik-dashboard-https:
      rule: "Host(`traefik.adm.local`)"
      entryPoints: ["https"]
      service: "api@internal"
      tls: {}
services:
  api@internal:
    loadBalancer:
      servers:
        - url: "http://traefik.adm.local:8080"

tls.yml

tls:
  stores:
    default:
      defaultCertificate:
        certFile: "C:/xampp/apache/conf/ssl.crt/server.crt"
        keyFile: "C:/xampp/apache/conf/ssl.key/server.key"
  certificates:
    - certFile: "C:/traefik/crt/traefik.adm.local/server.crt"
      keyFile: "C:/traefik/crt/traefik.adm.local/server.key"
      stores:
        - default
    - certFile: "C:/traefik/crt/web.adm.local/server.crt"
      keyFile: "C:/traefik/crt/web.adm.local/server.key"
      stores:
        - default

I also created a second site which a xampp dashboard for testing

web.yml

https:
  routers:
    web:
      rule: "Host(`web.adm.local`)"
      entryPoints: ["https"]
      service: "webhost"
      tls : {}

services:
  webhost:
    loadBalancer:
      servers:
        - url: "https://web.adm.local:444/dashboard/"

All urls are saved on the server's host file.

Both subdomain of adm.local returns the same error which relates to TLS handshake error and on browser it returns 404 page not found

Log:

2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:103 > Traefik version 3.2.1 built on 2024-11-20T18:10:43Z version=3.2.1
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/cmd/traefik/traefik.go:110 > Static configuration loaded [json] staticConfiguration={"api":{"dashboard":true},"entryPoints":{"http":{"address":":80","forwardedHeaders":{},"http":{"maxHeaderBytes":1048576,"redirections":{"entryPoint":{"permanent":true,"priority":9223372036854775806,"scheme":"https","to":"https"}}},"http2":{"maxConcurrentStreams":250},"transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s","readTimeout":"1m0s"}},"udp":{"timeout":"3s"}},"https":{"address":":443","forwardedHeaders":{},"http":{"maxHeaderBytes":1048576,"tls":{}},"http2":{"maxConcurrentStreams":250},"transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s","readTimeout":"1m0s"}},"udp":{"timeout":"3s"}}},"global":{},"log":{"format":"common","level":"DEBUG"},"providers":{"file":{"directory":"dynamic/","watch":true},"providersThrottleDuration":"2s"},"serversTransport":{"maxIdleConnsPerHost":200},"tcpServersTransport":{"dialKeepAlive":"15s","dialTimeout":"30s"}}
2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:626 >
Stats collection is disabled.
Help us improve Traefik by turning this feature on :)
More details on: https://doc.traefik.io/traefik/contributing/data-collection/

2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:231 > Starting TCP Server entryPointName=https
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:231 > Starting TCP Server entryPointName=http
2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:73 > Starting provider aggregator *aggregator.ProviderAggregator
2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *file.Provider
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *file.Provider provider configuration config={"directory":"dynamic/","watch":true}
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: dynamic/
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: dynamic/tls.yml
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: dynamic/traefik.yml
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: dynamic/web.yml
2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *traefik.Provider
2024-12-04T16:25:08+08:00 INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *acme.ChallengeTLSALPN
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *traefik.Provider provider configuration config={}
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{},"tcp":{},"tls":{"stores":{"default":{}}},"udp":{}} providerName=file
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *acme.ChallengeTLSALPN provider configuration config={}
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"middlewares":{"redirect-http-to-https":{"redirectScheme":{"permanent":true,"port":"443","scheme":"https"}}},"models":{"https":{"tls":{}}},"routers":{"http-to-https":{"entryPoints":["http"],"middlewares":["redirect-http-to-https"],"priority":9223372036854775806,"rule":"HostRegexp(`^.+$`)","ruleSyntax":"v3","service":"noop@internal"}},"serversTransports":{"default":{"maxIdleConnsPerHost":200}},"services":{"api":{},"dashboard":{},"noop":{}}},"tcp":{"serversTransports":{"default":{"dialKeepAlive":"15s","dialTimeout":"30s"}}},"tls":{},"udp":{}} providerName=internal
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/tls/certificate.go:132 > Adding certificate for domain(s) scms-dev.com,web.adm.local
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/tls/certificate.go:132 > Adding certificate for domain(s) scms-dev.com,traefik.adm.local
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_scheme.go:29 > Creating middleware entryPointName=http middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme routerName=http-to-https@internal
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/redirect/redirect_scheme.go:30 > Setting up redirection to https 443 entryPointName=http middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme routerName=http-to-https@internal
2024-12-04T16:25:08+08:00 DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:25 > Creating middleware entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
2024-12-04T16:25:12+08:00 DBG log/log.go:245 > http: TLS handshake error from 192.168.30.2:50520: remote error: tls: unknown certificate
2024-12-04T16:25:12+08:00 DBG log/log.go:245 > http: TLS handshake error from 192.168.30.2:50521: remote error: tls: unknown certificate
2024-12-04T16:25:27+08:00 DBG log/log.go:245 > http: TLS handshake error from 192.168.30.2:50553: remote error: tls: unknown certificate
2024-12-04T16:50:28+08:00 DBG log/log.go:245 > http: TLS handshake error from 127.0.0.1:50543: remote error: tls: unknown certificate
2024-12-04T16:50:32+08:00 DBG log/log.go:245 > http: TLS handshake error from 127.0.0.1:50548: remote error: tls: unknown certificate

A headstart where to debug will be great. Thanks!

2

How do you create valid and trusted TLS certs for .local?

3

A script with openssl is used to generate the certs

@echo off
set /p domain="Enter Domain: "

if not exist .\%domain% mkdir .\%domain%

"C:\Program Files\OpenSSL-Win64\bin\openssl.exe" req -config cert.conf -new -sha256 -newkey rsa:2048 -nodes -keyout %domain%\server.key -x509 -days 3650 -out %domain%\server.crt

echo.
echo -----
echo The certificate was provided.
echo.
pause

cert.conf

[ req ]

default_bits        = 2048
default_keyfile     = server-key.pem
distinguished_name  = subject
req_extensions      = req_ext
x509_extensions     = x509_ext
string_mask         = utf8only

[ subject ]

countryName                 = Country Name (2 letter code)
countryName_default         = US

stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = NY

localityName                = Locality Name (eg, city)
localityName_default        = New York

organizationName            = Organization Name (eg, company)
organizationName_default    = Example, LLC

commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_default          = scms-dev.com

emailAddress                = Email Address
emailAddress_default        = test@example.com

[ x509_ext ]

subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid,issuer

basicConstraints       = CA:FALSE
keyUsage               = digitalSignature, keyEncipherment
subjectAltName         = @alternate_names
nsComment              = "OpenSSL Generated Certificate"

[ req_ext ]

subjectKeyIdentifier = hash

basicConstraints     = CA:FALSE
keyUsage             = digitalSignature, keyEncipherment
subjectAltName       = @alternate_names
nsComment            = "OpenSSL Generated Certificate"

[ alternate_names ]

DNS.1       = scms-dev.com
1 Like
4

How do you run Traefik? I assume it’s a Docker container? Then you would need Linux paths and bind mount the files accordingly.

In your scenario you have potentially two sets of TLS certs.

  1. The one for entrypoint / router which is used for client/browser connections. The client needs to know and trust the cert, did you import the custom public key on client for .local ?
  2. The one used with the https target service. With the .local domain it’s not an official cert, so you need to provide the public key or set insecureSkipVerify globally or create a transport with it and assign it to the service.
5

As stated on the environment, I am currently using Binary file exe on Windows to run Traefik

For your questions

  1. I didn't import the install the certs yet. Is it required?
  2. I already assigned insecureSkipVerify to true on my traefik.yml
6

I think the issue is that your browser/client accessing Traefik is not trusting the TLS cert, because the cert authority of the custom certs is unknown. You probably need to import the certs into the OS trust store or add them to the client/browser.

A way around is to use regular domains, like traefik.local.example.com. You can then use LetsEncrypt dnsChallenge to still generate a valid trusted TLS cert, even when the domain is not reachable from Internet. You can even set your internal target IP on the (external) DNS server.

7

I'll try that OS trust store. If not, I will switch to more conventional domains

8

Tried installing it to Trust Store Certification Authorities with traefik.nic.org domain and it still the same error.

Maybe there's a problem on my config files?

9

I somehow fixed the warn about TLS handshake error. But page still shows 404 page not found. I think it is not routing the traffic if the client is accessing the domain traefik.nic.org to the localhost port 444 where the website is currently being served.

P.S.
Traefik is using 8080, 80, and 443 ports and on the same server, I have a xampp instance where I changed ports to 81 and 444

10

You need a domain to resolve to an IP and the port needs to be open for Traefik to be accessed.

Enable and check Traefik debug log (doc) and Traefik access log in JSON format (doc). Are requests showing up?

For TLS to work you would usually buy a TLS cert or use LetsEncrypt for a public domain (at least public root domain). For those the CA chain cert is present in OS and requests to sites with such cert will be trusted by client.

The TLS errors you see are just a debug info (DBG), some client is trying to connect without the matching CA chain cert installed. On public Internet this happens all the time when some bots try to access.

If you create your own custom cert, you need to make sure you have the CA chain cert imported into OS or the browser. This is of course not a workable solution for public sites.

Note that loadbalancer only uses protocol and domain, not the path. Path is taken from original request unless rewritten with middlewares.

Finally note that you can’t change Traefik dashboard to be accessed by / only (removing /dashboard).

11

Traefik and my xampp instance is both running on the same server. If my xampp is running on port 444. Does it need to be open? It should work like a reverse proxy right? and direct the traffic to that specific on port on localhost

12

If you want to proxy all traffic through Traefik, then the port of XMPP does not have to be open externally.

Usually Traefik and target service share an internal Docker network, see simple Traefik example. Within all ports are reachable.

13

Is it not possible to use traefik without docker? I am hosting my services on standalone and without docker instance

14

Of course, it is possible, I wrote "usually".

15

I see, I've been diagnosing this problem for a week now

16

You stated you got the TLS part solved.

What URL do you request? What is logged during a request?

17

Somehow, I fixed the connection my configuration by combining all my configuration routers in one dynamic.yml

18

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.