Hi, My current setup is working well, but requesting certificates is an issue.
I am running a 10 node HashiCorp Nomad + Consul cluster. Each machine is running keepalived.
Each machine is also running an instance of Traefik as a Nomad job, the idea being when a host fails, the vip gets updated and a new Traefik instance takes over (Acts like a hot standby).
I'm persisting the acme.json file on each host, so new certificate requests aren't performed on reboots etc.
However when I add a new service, there is a certificate request per host (so 10 in total here) so I am hitting rate limiting.
My question is:
Is there a way to share the certificates amongst Traefik instances (either by external service - please recommend any)
Or is there a preferred setup for hot standby?
Distributed LetsEncrypt is a feature of paid "Traefik Enterprise" (doc).
It is possible to use for example certbot behind Traefik to create the certs [1, 2] and to share them with all Traefik instances, via http or a shared folder.
We are currently still evaluating, as we don’t want to introduce another potential breaking point when the internal network connection to the cert source fails. We want the certs automatically distributed to and persisted on all nodes.
Are there any workarounds? My requirements are automatic service discovery from consul and certificate provisioning.
The only things I've found (and what you mentioned) are super ad-hoc
But it seems like all the big players (NGINX, HAProxy, Traefik etc) are all pay-walling these features.
The jump from free to tens of thousands of dollars seems a little too steep.
Kubernetes has cert manager as free solution, Docker Swarm has no free solution AFAIK, not sure about nomad.
As a "manual" workaround: either you use a separate service to create the certs and supply them to your Traefik instances.
Or you setup only one Traefik instance with LE and distribute the certs to the other instances. Probably need to export them as plain TLS certs from acme.json (using a tool).
For both cases use a dynamic config file for the certs, watch the file in static config, touch for reload.
UPDATE: if you are already using consul as "shared config storage", then its probably easier to have one Traefik create the certs and export them to consul, have other Traefik instances load from consul.