How to use my own certificates into the configmap configuration. K8s

titansmc · October 29, 2019, 5:10pm

Hi,
I have been through the documentation and at some point they state "kubernetes users must provide certificates through secrets", but, how do I achieve this?
I have a ConfigMap with the configuration I want to apply to traefik:

    [tls]
      [tls.certificates] 
        stores = ["default"]
    [tls.stores]
      [tls.stores.default]
        [tls.stores.default.defaultCertificate]
          certFile =
          keyFile  =

But now I don't know how to present the certificate to it. Basically I have a wildcard certificate that I would like to apply to all the configurations by default.

Thanks.

dduportal · October 31, 2019, 12:33pm

Hi @titansmc, you can check Can I set namespace for TLS secretName in IngressRoute? , the solution is the same as for you

TL;DR;

Instead of setting tls: secretName on each IngressRoute (which implies having the secret on each namespace),
Mount the secret in Traefik
Enable fileprovider and specify it the path to the mounted ConfigMap containing the dynamic file config you provided on your message
Set the directive tls: {} to the IngressRoutes

titansmc · October 31, 2019, 12:53pm

Hi @dduportal, thanks, I did all the static and dynamic config and it looks fine to me, but I am using kubernetes ingress but not the CRD ingressroute and it seems it ingress doesn't accept it.

dduportal · October 31, 2019, 12:57pm

Which version of Traefik are you using?

If you are using v2.0, it is recommend to switch to CRD IngressRoute as v2 does not support annotations.

However what you want to achieve should still work, if you enable the directive tls and specify only the hosts (please look, at the official Kubernetes documentation https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). Don't specify any secretName and it should be ok.

Let us know and share some examples with us if it does not work.

titansmc · October 31, 2019, 1:00pm

Hi,
Thanks, I would then consider switching to IngresRoute ( Actually I switched back because I want to use official helm charts and I don't know a way to get it working with CRD )
It seems the problem comes now with:

level=error msg="Cannot start the provider *file.Provider: toml: cannot load TOML value of type map[string]interface {} into a Go slice"

BTW traefik 2.0.2

dduportal · October 31, 2019, 1:02pm

Can you share the Traefik's deployment file, and the full configmap mapping to traefik.toml please? As this is only partial information, but it looks like there is a syntax error in your TOML file.

titansmc · October 31, 2019, 1:06pm

Sure.
I can see the certs properly mounted in the container, and also both files. (Static and dynamic)

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: traefik
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v2.0.2
        name: traefik-ingress-lb
        ports:
        - name: web
          containerPort: 80
          protocol: TCP
        - name: websecure
          containerPort: 443
          protocol: TCP
        - name: admin
          containerPort: 8080
          protocol: TCP
        volumeMounts:
          - mountPath: "/certs/my.de/cert1.pem"
            name: my-cert
            subPath: cert1.pem
            readOnly: true
          - mountPath: "/certs/my.de/privkey1.pem"
            name: my-key
            subPath: privkey1.pem
            readOnly: true
          - mountPath: "/etc/traefik/dynamic.toml"
            name: traefik-dynamic-configmap
            subPath: dynamic.toml
            readOnly: true
          - mountPath: "/etc/traefik/traefik.toml"
            name: traefik-toml-configmap
            subPath: traefik.toml
            readOnly: true
        #args:
        #- --configfile=/config/traefik.toml
        #- --logLevel=DEBUG
      volumes:
        - name: traefik-dynamic-configmap
          configMap:
            name: traefik-dynamic-configmap
        - name: traefik-toml-configmap
          configMap:
            name: traefik-toml-configmap
        - name: my-cert
          secret:
            secretName: my-cert
        - name: my-key
          secret:
            secretName: my-key

and here is the configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-toml-configmap
  namespace: traefik
data:
  traefik.toml: |
    #[global]
    #
    [accessLog] 

    [serversTransport]
    # Do not verify backend certificates (use https backends)
      insecureSkipVerify = true
    
    [entryPoints]
      [entryPoints.web]
        address = ":80"
        #compress = true
      [entryPoints.websecure]
        address = ":443"
        [entryPoints.websecure.forwardedHeaders]
          insecure = true
      [entryPoint.traefik]
        address = ":8080"
    
    [api]
      insecure = true
      dashboard = true
      debug = true
    
    [log]
      level = "DEBUG"
    
    [providers]
      [providers.file]
        filename = "/etc/traefik/dynamic.toml"
        watch = "true"
      [providers.kubernetesingress]

And the dynamic one

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-dynamic-configmap
  namespace: traefik
data:
  dynamic.toml: |
    [tls.options]

      [tls.options.default]
        sniStrict = true
        minVersion = "VersionTLS12"
        cipherSuites = [
          "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
        ]

    [tls.certificates]
      certFile = "/certs/my.de/cert1.pem"
      keyFile  = "/certs/my.de/privkey1.pem"
      stores = ["default"]       

    [tls.stores]
      [tls.stores.default]
        [tls.stores.default.defaultCertificate]
          certFile = "/certs/my.de/cert1.pem"
          keyFile  = "/certs/my.de/privkey1.pem"

titansmc · October 31, 2019, 1:20pm

I think the issue was the main [tls] missing. Now it works:

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-dynamic-configmap
  namespace: traefik
data:
  dynamic.toml: |
    [tls]
      [tls.options]

        [tls.options.default]
          sniStrict = true
          minVersion = "VersionTLS12"
          cipherSuites = [
            "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
          ]

      [[tls.certificates]]
        certFile = "/certs/my.de/cert1.pem"
        keyFile  = "/certs/my.de/privkey1.pem"
        stores = ["default"]       

      [tls.stores]
        [tls.stores.default]
          [tls.stores.default.defaultCertificate]
            certFile = "/certs/my.de/cert1.pem"
            keyFile  = "/certs/my.de/privkey1.pem"

dduportal · October 31, 2019, 2:06pm

The "main" tls is not mandatory for information.

But the issue was the fact that tls.certificates required 2 pairs of brackets (and not only one),
to express it's an element of a collection (and not a single directive).

[tls.certificates] -> [[tls.certificates]].

In your latest message, you fixed it, it's why it is working

titansmc · November 5, 2019, 5:40pm

With this configuration, I can access services that uses the default store but not the ones that should be using opsk1 store. ( I am assuming that when typing https://grafana.ops-k1.domain.de it will use the appropriate certificate)

    [tls]
      [tls.options]

        [tls.options.default]
          sniStrict = true
          minVersion = "VersionTLS12"
          cipherSuites = [
            "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
          ]

      [[tls.certificates]]
        certFile = "/certs/domain.de/cert1.pem"
        keyFile  = "/certs/domain.de/privkey1.pem"
        stores = ["default"]       

      [[tls.certificates]]
        certFile = "/certs/ops-k1.domain.de/cert1.pem"
        keyFile  = "/certs/ops-k1.domain.de/privkey1.pem"
        stores = ["opsk1"]

      [tls.stores]
        [tls.stores.default]
          [tls.stores.default.defaultCertificate]
            certFile = "/certs/domain.de/cert1.pem"
            keyFile  = "/certs/domain.de/privkey1.pem"
        [tls.stores.opsk1]

Also the documentation about stores is a bit confusing, since it says that :

Any store definition other than the default one (named default) will be ignored, and there is thefore only one globally available TLS store.

but then in the File dynamic configuration you can see examples with:

  [tls.stores]
    [tls.stores.Store0]
      [tls.stores.Store0.defaultCertificate]
        certFile = "foobar"
        keyFile = "foobar"
    [tls.stores.Store1]
      [tls.stores.Store1.defaultCertificate]
        certFile = "foobar"
        keyFile = "foobar"

Those are the logs, which I am not sure I understand:

time="2019-11-05T17:15:46Z" level=debug msg="No default certificate, generating one" tlsStoreName=opsk1
time="2019-11-05T17:15:46Z" level=debug msg="Adding certificate for domain(s) *.domain.de"
time="2019-11-05T17:15:46Z" level=debug msg="Adding certificate for domain(s) *.ops-k1.domain.de"
time="2019-11-05T17:15:46Z" level=debug msg="No store is defined to add the certificate MIIKWDCCCUCgAwIBAgIMISGefWrCZCQykHsfSqMA0GCSqGSIb3DQ, it will be added to the default store."

titansmc · November 5, 2019, 6:24pm

I see what is going on. I had enabled strict SNI and since it tries to serve *.domain.de it doesn't work. even I have *.ops-k1.domain.de certificate too.

Any idea why Traefik cannot pick up the right one?

titansmc · November 5, 2019, 6:35pm

Now I have added the *.ops-k1.domain.de certificate to default store and it works, but what I still don't get is how handle different certs in different stores.

dduportal · November 13, 2019, 10:24am

Hi @titansmc, is there anything blocking you to only use the default store, in which you add all certificates?

titansmc · November 13, 2019, 10:26am

@dduportal not really, is just that I understood somehow that different stores were designed to split certificates. But I can definitely use this single store. I still would like to know why several stores are available but only the default can be used.

Thanks.

dduportal · November 13, 2019, 10:34am

I cannot answer this as I don't know either.

My guess is that this is a foreseen feature, that required the Traefik foundation to be ready to handle it,
but the elements to expose this feature to end user have not been done (for whatever reason: no time, no opportunity, no stability, whatever).

By the way, the section "Reference" of the documentation is generated manually: it explains why you see "excerpts" of configuration.

zespri · November 13, 2019, 7:58pm

My guess that is has to do with the paid version. In the Enterprise version you'll be able to have more KV stores, such as Consul or Etcd, which would be separate from file. The community version is not getting them, so one store only. To repeat - this is a guess.

Topic		Replies	Views
Traefik - Custom TLS certificate Traefik v2 kubernetes-ingress	4	275	July 19, 2024
Can I set namespace for TLS secretName in IngressRoute? Traefik v2 kubernetes-crd	4	7643	October 29, 2019
How to configure TLS with CLI Traefik v2 cli	6	1467	November 20, 2019
Setup TLS/SSL, need help Traefik v2 kubernetes-ingress	3	755	April 16, 2021
Minimal but meaningful v2.0 GKE configuration Traefik v2 kubernetes-crd , kubernetes-ingress	0	676	August 12, 2019

How to use my own certificates into the configmap configuration. K8s

Related topics