Traefik mTLS ClientCA certificates?

Traefik Traefik v1
1

The Traefik v1.7 documentation states:

TLS Mutual Authentication¶
TLS Mutual Authentication can be optional or not.

If optional = true, if a certificate is provided, verifies if it is signed by a specified Certificate Authority (CA). Otherwise proceeds without any certificate.
If optional = false, Traefik will only accept clients that present a certificate signed by a specified Certificate Authority (CA).
ClientCAFiles can be configured with multiple CA:s in the same file or use multiple files containing one or several CA:s. The CA:s has to be in PEM format.

I've created a test self-signed certificate, converted it to the .pem format and I'm getting the following log output from traefik which is crashing:

PEM

-----BEGIN CERTIFICATE-----
(base64)
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
(base64)
-----END PRIVATE KEY-----

Logs

{"level":"info","msg":"Using TOML configuration file /config/traefik.toml","time":"2019-07-26T13:12:45Z"}
{"level":"info","msg":"No tls.defaultCertificate given for https: using the first item in tls.certificates as a fallback.","time":"2019-07-26T13:12:45Z"}
{"level":"info","msg":"Traefik version v1.7.12 built on 2019-05-29_07:35:02PM","time":"2019-07-26T13:12:45Z"}
{"level":"info","msg":"\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/basics/#collected-data\n","time":"2019-07-26T13:12:45Z"}
{"level":"info","msg":"Preparing server http \u0026{Address::80 TLS:\u003cnil\u003e Redirect:0xc00018a200 Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:true ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc0001b9440} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2019-07-26T13:12:45Z"}
{"level":"info","msg":"Preparing server https \u0026{Address::443 TLS:0xc000422990 Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:true ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc0001b93c0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2019-07-26T13:12:45Z"}
{"level":"fatal","msg":"Error preparing server: error creating TLS config: invalid certificate(s) in /mtls/clientCaCert-0.crt","time":"2019-07-26T13:12:45Z"}

Q1: I believe I may have messed something up at the certificate creation/transformation level
Can you recommend what is the proper way to have this setup with OpenSSL ?

Q2: There's currently a PR open which is aiming to add mTLS support to the helm chart.


I believe @daniel.tomcej was reviewing this

One of the code fragments inside is:

{{- define "traefik.ssl.mtls.clientCAs" -}}
         files = [
	   {{- range $idx, $_ := .Values.ssl.mtls.clientCaCerts }}
	     {{- if $idx }}, {{ end }}
	     {{- printf "/mtls/clientCaCert-%d.crt" $idx | quote }}
	   {{- end -}}
         ]
{{- end -}}

The above fragment renders .crt files - doesn't that violate the PEM format requirement?

2

fixed ordering after the edits, sorry about the mess

3

Hello @mmisztal1980,

You are close, but I would like to provide clarification on a few points:

  1. PEM formatting is chosen because it is an easier standard to work with as opposed to DER.

The PEM format for a full trust chain is:

-----BEGIN PRIVATE KEY----- 
(Your Private Key: name.key) 
-----END PRIVATE KEY----- 
-----BEGIN CERTIFICATE----- 
(Your Primary SSL certificate: name.crt) 
-----END CERTIFICATE----- 
-----BEGIN CERTIFICATE----- 
(Your Intermediate certificate: Intermediate.crt) 
-----END CERTIFICATE----- 
-----BEGIN CERTIFICATE----- 
(Your Root certificate: Root.crt) 
-----END CERTIFICATE-----

In PEM format, if a key is included it needs to be first, and every certificate has to be signed by the following certificate.

  1. PEM format is the default exporting format for Openssl, and many Registrars, so 99.99% of people don't have to do any conversion, however, they often have to combine their intermediate certificates into the PEM file.

Nope! Not at all.

1 Like
4

Thanks @daniel.tomcej that clears up a few things. What about self-signed certificates? I take it the full trust chain is not required?

5

@mmisztal1980, In the case of a self-signed certificate, it signs itself, (meaning it is a root certificate).

1 Like
6

I've reordered the contents of my .pem certificate file as suggested:

-----BEGIN PRIVATE KEY----- 
(My self signed certificate key: certificate.key | base64) 
-----END PRIVATE KEY----- 
-----BEGIN CERTIFICATE----- 
(My self signed certificate: certificate.crt | base64) 
-----END CERTIFICATE-----

Still getting:

{"level":"fatal","msg":"Error preparing server: error creating TLS config: invalid certificate(s) in /mtls/clientCaCert-0.crt","time":"2019-07-26T15:10:34Z"}
7

@mmisztal1980,

Do not provide your private key, it is not needed.

The CA certificate in that configuration is just to verify that it is used as a signing authority.

8

Still fails without the private key :confused: not sure what else might be wrong. The cert was generated using:

openssl req -new -newkey rsa:4096 -x509 -sha256 -days 1825 -nodes -out certificate.crt -keyout certificate.key

At the moment I'm attempting so use the contents of the .crt file :

-----BEGIN CERTIFICATE----- 
(My self signed certificate: certificate.crt | base64) 
-----END CERTIFICATE-----
9

@mmisztal1980, The certificate content should already be in base64.

OpenSSL already adds the header and trailer to the crt file.

You should just be able to use the certificate.crt directly, without any modifications.

10

That is exactly what I'm trying to do.

Here's my values.yaml override:

ssl:
  enabled: true
  enforced: true
  permanentRedirect: true
  defaultCert: (...)
  defaultKey: (...)
  mtls:
    enabled: true
    optional: true
    clientCaCerts:
    - <certificate.crt contents inc. header & trailer>
11

@daniel.tomcej Is there any way I can get more diagnostic information than with:

debug:
  enabled: true

?

12

@daniel.tomcej still trying ,no luck - I tried stripping the prefix/suffix but there was no observable effect either. Is there any way to get a more detailed error message?

13

I am having exactly the same problem.

Any news on this? Is there a way to more thoroughly debug this and understand what the certificate needs to look like to work in traefik?