Let's Encrypt fails with 400 Connection Refused

Traefik Traefik v2 (latest)
1

Hello,

my dockerized Traefik installation used to run just fine, but recently, renewal of let's encrypt certificates fails:

traefik    | time="2021-02-17T18:50:38Z" level=error msg="Error renewing certificate from LE: {traefik.host.de []}, error: one or more domains had a problem:\n[traefik.host.de] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Connection refused, url: \n" providerName=le.acme

(from docker-compose logs)
What looks suspicious to me is the newline \n in the error message.

with enabling debug output I get

traefik    | time="2021-02-17T19:50:13Z" level=debug msg="legolog: [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/123"
traefik    | time="2021-02-17T19:50:13Z" level=error msg="Unable to obtain ACME certificate for domains \"cloud.host.de\": unable to generate a certificate for the domains [cloud.host.de]: error: one or more domains had a problem:\n[cloud.host.de] acme: error: 400 :: urn:ietf:params:acme:error:connection :: Connection refused\n" providerName=le.acme routerName=nextcloud@docker rule="Host(`cloud.host.de`)"
traefik    | time="2021-02-17T19:50:13Z" level=debug msg="legolog: [INFO] Unable to deactivate the authorization: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/123"

(changed to the staging server and removed the letsencrypt directory to force generating new certificates)

The (redacted) docker file for Traefik is:

version: "3.3"

services:
  traefik:
    image: "traefik:v2.4"
    container_name: "traefik"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "./traefik.yaml:/etc/traefik/traefik.yaml"
      - "./dynamic_conf.yaml:/etc/traefik/dynamic_conf.yaml"
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`traefik.host.de`)"
      - "traefik.http.routers.api.service=api@internal"
      - "traefik.http.routers.api.entrypoints=https"
      - "traefik.http.routers.api.tls.certresolver=le"

The relevant portion of traefik.yaml

certificatesResolvers:
  le:
    acme:
      tlschallenge: true
      email: "me@host.de"
      storage: "/letsencrypt/acme.json"

The host is on a static IP and can be reached just fine from the internet, e.g. the Traefik dashboard and also all other domains can be reached.

What can be the problem here?

Thanks!

2

Hello @Horus

Just the basic question before moving deeper with debugging - can you please make sure that your host is reachable and the port 443 is not being blocked?

Thank you,

3

Hi,
I would just join the thread here, because my fresh docker-based installation has the exact same problems.
Configurations seem to be quite identical. My port is reachable.

The certificate generation using the http-challenge works out of the box. I can reach all of my hosts using https but using the TLS challenge would be great. Debug output is the same as above.

I can post my simple setup later but seems to be quite similar to @Horus config.

Cheers and thanks
Dennis

4

Same for me, 443 is reachable, TLS (https on 443) works just fine, because the certificates are still valid, but renewal doesn't work.

I got it working now by switching to httpChallenge.

5

So I just played around and maybe I found something.

I removed my AAAA DNS record and the tlsChallenge worked again. I am afraid, that there is some issue with my IPv6 configuration. As a matter of fact I didn't config anything with IPv6 which probably means, that Let'sEncrypt just tries to connect to my v6 address and my server does not respond properly.

Is there anything, I need to keep In mind, using IPv6? Do I need to configure my Docker-installation somehow or tell traefik to respond to v6-requests? I defined my addresses in the traefik.toml as follows

[entryPoints]
  [entryPoints.web]
    address = ":80"

  [entryPoints.websecure]
    address = ":443"

which I thought was enough.
But my docker ps -a shows something like

0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp

so maybe it is something docker and/or IPv6-related?

(sorry I am not that long into the server-game :smiley: )

6

So I played around and finally think that enabling ipv6 for docker is the solution here.

You need to edit the /etc/docker/daemon.json according to https://docs.docker.com/config/daemon/ipv6/)

Edit /etc/docker/daemon.json , set the ipv6 key to true and the fixed-cidr-v6 key to your IPv6 subnet. In this example we are setting it to 2001:db8:1::/64 .

{
"ipv6": true,
"fixed-cidr-v6": "2001:db8:1::/64"
}

IPv6 is now enabled for the default bridge network. I have created my own network with IPv6 support with

docker network create --ipv6 --subnet=2001:db8:1::/112 web

where I connect all my traefik-related containers. By default the containers are now reachable with IPv6 as well as IPv4

docker ps -a
> 2aa1bf538b47   traefik:latest   "/entrypoint.sh trae…"   11 minutes ago   Up 11 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   traefik

So I can keep my AAAA record for my domain AND the TLS challenge works again out of the box. The certificates are generated without any problems.

Maybe there are some more professional exlainations. This solution works for my setup :slight_smile:

Cheers and thanks,
Dennis

1 Like