Retrieve wildcard certificate for subdomain with TransIP as DNS provider

What I try to do

I am trying to get a wildcard certificate for a subdomain like *.marktest.dev.mobilea.nl for a registered domain with TransIP using the DNS challenge, since I use internal IP addresses to get certificated for. The domain that is registered would be mobilea.nl.

Also I am using the tutorial from DockerSwarm.rocks with some modification to use the DNS challenge instead.

DNS settings mobilea.nl

Here the related DNS settings from the TransIP panel, which are correctly configured, since I have more of these configurations working on the domain for other purposes.:

Name TTL Type Value
marktest.dev 1 min. A 172.22.22.5
*.marktest.dev 1 min. A 172.22.22.5

Modifications docker-compose.yml from tutorial DockerSwarm.rocks

The docker secret is created earlier and the environment variables are set correctly.

  traefik:
   ...
    secrets:
      - source: transip_key
    command: >
      ...
      # --acme.httpChallenge.entryPoint=http
      --acme.dnsChallenge.provider="transip"
      #--acme.onhostrule=true
      --acme.domains="*.marktest.dev.mobilea.nl"
      ...
      -logLevel=DEBUG
      ...
    environment:
      - TRANSIP_ACCOUNT_NAME=${TRANSIP_ACCOUNT_NAME?Variable TRANSIP_ACCOUNT_NAME not set}
      - TRANSIP_PRIVATE_KEY_PATH=${TRANSIP_PRIVATE_KEY_PATH?Variable TRANSIP_PRIVATE_KEY_PATH not set}

Filtered output on acme

6time="2019-07-23T09:08:48Z" level=debug msg="Global configuration loaded {\"LifeCycle\":{\"RequestAcceptGraceTimeout\":0,\"GraceTimeOut\":10000000000},\"GraceTimeOut\":0,\"Debug\":false,\"CheckNewVersion\":true,\"SendAnonymousUsage\":false,\"AccessLogsFile\":\"\",\"AccessLog\":{\"format\":\"common\"},\"TraefikLogsFile\":\"\",\"TraefikLog\":null,\"Tracing\":null,\"LogLevel\":\"DEBUG\",\"EntryPoints\":{\"http\":{\"Address\":\":80\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"https\":{\"Address\":\":443\",\"TLS\":{\"MinVersion\":\"\",\"CipherSuites\":null,\"Certificates\":[],\"ClientCAFiles\":null,\"ClientCA\":{\"Files\":null,\"Optional\":false},\"DefaultCertificate\":null,\"SniStrict\":false},\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"traefik\":{\"Address\":\":8080\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}}},\"Cluster\":{\"Node\":\"0baf9f6b-eb5f-42f4-8f56-95c53536e77b\",\"Store\":{\"Store\":{},\"Prefix\":\"traefik\"}},\"Constraints\":[\"tag==traefik-public\"],\"ACME\":{\"Email\":\"mark@mobilea.nl\",\"Domains\":[{\"Main\":\"*.marktest.dev.mobilea.nl\",\"SANs\":null}],\"Storage\":\"traefik/acme/account\",\"StorageFile\":\"\",\"OnDemand\":false,\"OnHostRule\":true,\"CAServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"EntryPoint\":\"https\",\"KeyType\":\"\",\"DNSChallenge\":{\"Provider\":\"transip\",\"DelayBeforeCheck\":10000000000,\"Resolvers\":null,\"DisablePropagationCheck\":false},\"HTTPChallenge\":null,\"TLSChallenge\":null,\"DNSProvider\":\"\",\"DelayDontCheckDNS\":0,\"ACMELogging\":true,\"OverrideCertificates\":false,\"TLSConfig\":null},\"DefaultEntryPoints\":[\"http\"],\"ProvidersThrottleDuration\":2000000000,\"MaxIdleConnsPerHost\":200,\"IdleTimeout\":0,\"InsecureSkipVerify\":false,\"RootCAs\":null,\"Retry\":null,\"HealthCheck\":{\"Interval\":30000000000},\"RespondingTimeouts\":null,\"ForwardingTimeouts\":null,\"AllowMinWeightZero\":false,\"KeepTrailingSlash\":false,\"Web\":null,\"Docker\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":null,\"Trace\":false,\"TemplateVersion\":2,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"unix:///var/run/docker.sock\",\"Domain\":\"\",\"TLS\":null,\"ExposedByDefault\":false,\"UseBindPortIP\":false,\"SwarmMode\":true,\"Network\":\"\",\"SwarmModeRefreshSeconds\":15},\"File\":null,\"Marathon\":null,\"Consul\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"consul-leader:8500\",\"Prefix\":\"traefik\",\"TLS\":null,\"Username\":\"\",\"Password\":\"\"},\"ConsulCatalog\":null,\"Etcd\":null,\"Zookeeper\":null,\"Boltdb\":null,\"Kubernetes\":null,\"Mesos\":null,\"Eureka\":null,\"ECS\":null,\"Rancher\":null,\"DynamoDB\":null,\"ServiceFabric\":null,\"Rest\":null,\"API\":{\"EntryPoint\":\"traefik\",\"Dashboard\":true,\"Debug\":false,\"CurrentConfigurations\":null,\"Statistics\":null},\"Metrics\":null,\"Ping\":null,\"HostResolver\":null}",
time="2019-07-23T09:08:49Z" level=debug msg="Building ACME client...",
time="2019-07-23T09:08:49Z" level=info msg="Starting ACME renew job...",
time="2019-07-23T09:08:50Z" level=debug msg="No ACME certificate to generate for domains [\"consul.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:08:50Z" level=debug msg="No ACME certificate to generate for domains [\"traefik.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:08:50Z" level=info msg="Retrieving ACME certificates...",
time="2019-07-23T09:08:50Z" level=debug msg="Loading ACME certificates [*.marktest.dev.mobilea.nl]...",
time="2019-07-23T09:08:50Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Obtaining bundled SAN certificate",
time="2019-07-23T09:08:50Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/xjVymJwDH30c3UwvTBWE8kOBlCk_--0Cci-rIwoe04w",
time="2019-07-23T09:08:50Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: use dns-01 solver",
time="2019-07-23T09:08:50Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Preparing to solve DNS-01",
time="2019-07-23T09:08:53Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Trying to solve DNS-01",
time="2019-07-23T09:08:53Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Checking DNS record propagation using [127.0.0.11:53]",
time="2019-07-23T09:09:11Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Cleaning DNS-01 challenge",
time="2019-07-23T09:09:13Z" level=info msg="legolog: [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz/xjVymJwDH30c3UwvTBWE8kOBlCk_--0Cci-rIwoe04w",
time="2019-07-23T09:09:13Z" level=error msg="Error getting ACME certificate for domain [\"*.marktest.dev.mobilea.nl\"]: cannot obtain certificates: acme: Error -> One or more domains had a problem:\n[*.marktest.dev.mobilea.nl] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: No TXT record found at _acme-challenge.marktest.dev.mobilea.nl, url: \n",
time="2019-07-23T09:09:13Z" level=info msg="Retrieved ACME certificates",
time="2019-07-23T09:09:13Z" level=debug msg="No ACME certificate to generate for domains [\"consul.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:09:13Z" level=debug msg="No ACME certificate to generate for domains [\"traefik.marktest.dev.mobilea.nl\"]."

What i would expect?

I would expect the TXT records are created and validated so the certificate would be created.

Any ideas what I am doing wrong here?

Hi @mark-veenstra !

Do you have the same behavior if you:

  • Set Traefik replicas to only 1 (export TRAEFIK_REPLICAS=1)
  • And disable Consul while setting ACME to use an acme.json file instead

?

Also, could you give us a full reproduction case here? As we are not in charge of Docker-Swarm.rocks, you should double-check some configuration (ref. https://github.com/tiangolo/dockerswarm.rocks/pull/22).

If I understand you correctly the TRAEFIK_REPLICAS should be set to 1 and the CONSUL_REPLICAS to 0. That's how it is deployed already if that's what you are asking.

The be complete, the following deploy script is used:

#!/bin/bash

docker network create --driver=overlay traefik-public || true
export EMAIL="mark@mobilea.nl"
export DOMAIN="marktest.dev.mobilea.nl"

# Traefik / Consul admin login
export USERNAME=<ADMINUSER>
export PASSWORD=<ADMINPASS>
export HASHED_PASSWORD=$(openssl passwd -apr1 $PASSWORD)

export CONSUL_REPLICAS=0
export TRAEFIK_REPLICAS=1

# Let's Encrypt DNS challenge
# Here I create the file `transip_key`
docker secret create transip_key transip_key || true
export TRANSIP_ACCOUNT_NAME="<ACCOUNT_NAME>"
export TRANSIP_PRIVATE_KEY_PATH="/run/secrets/transip_key"

docker stack deploy -c docker-compose.yml traefik-consul

Here an overview of the containers from Portainer:

@dduportal I just missed your reply on acme.json. I now switched it also to use the acme.json as storage by changing:

--acme.storage="traefik/acme/account"

to

--acme.storage="acme.json"

But the same output is there:

time="2019-07-23T09:37:06Z" level=debug msg="No ACME certificate to generate for domains [\"consul.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:37:06Z" level=debug msg="No ACME certificate to generate for domains [\"traefik.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:37:06Z" level=info msg="Retrieved ACME certificates",
time="2019-07-23T09:37:06Z" level=error msg="Error getting ACME certificate for domain [\"*.marktest.dev.mobilea.nl\"]: cannot obtain certificates: acme: Error -> One or more domains had a problem:\n[*.marktest.dev.mobilea.nl] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: No TXT record found at _acme-challenge.marktest.dev.mobilea.nl, url: \n",
time="2019-07-23T09:36:45Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Checking DNS record propagation using [127.0.0.11:53]",
time="2019-07-23T09:37:02Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Cleaning DNS-01 challenge",
time="2019-07-23T09:37:06Z" level=info msg="legolog: [INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz/dUxHvmUsOipPUtN96lB-Ekq70LU15IAT6WMxoDqno9E",
time="2019-07-23T09:36:45Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Trying to solve DNS-01",
time="2019-07-23T09:36:43Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Preparing to solve DNS-01",
time="2019-07-23T09:36:43Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: use dns-01 solver",
time="2019-07-23T09:36:42Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] acme: Obtaining bundled SAN certificate",
time="2019-07-23T09:36:43Z" level=info msg="legolog: [INFO] [*.marktest.dev.mobilea.nl] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/dUxHvmUsOipPUtN96lB-Ekq70LU15IAT6WMxoDqno9E",
time="2019-07-23T09:36:42Z" level=debug msg="Loading ACME certificates [*.marktest.dev.mobilea.nl]...",
time="2019-07-23T09:36:42Z" level=info msg="Retrieving ACME certificates...",
time="2019-07-23T09:36:42Z" level=debug msg="No ACME certificate to generate for domains [\"consul.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:36:42Z" level=debug msg="No ACME certificate to generate for domains [\"traefik.marktest.dev.mobilea.nl\"].",
time="2019-07-23T09:36:42Z" level=info msg="legolog: [INFO] acme: Registering account for mark@mobilea.nl",
time="2019-07-23T09:36:42Z" level=debug msg="Building ACME client...",
+time="2019-07-23T09:36:39Z" level=debug msg="Global configuration loaded {\"LifeCycle\":{\"RequestAcceptGraceTimeout\":0,\"GraceTimeOut\":10000000000},\"GraceTimeOut\":0,\"Debug\":false,\"CheckNewVersion\":true,\"SendAnonymousUsage\":false,\"AccessLogsFile\":\"\",\"AccessLog\":{\"format\":\"common\"},\"TraefikLogsFile\":\"\",\"TraefikLog\":null,\"Tracing\":null,\"LogLevel\":\"DEBUG\",\"EntryPoints\":{\"http\":{\"Address\":\":80\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"https\":{\"Address\":\":443\",\"TLS\":{\"MinVersion\":\"\",\"CipherSuites\":null,\"Certificates\":[],\"ClientCAFiles\":null,\"ClientCA\":{\"Files\":null,\"Optional\":false},\"DefaultCertificate\":null,\"SniStrict\":false},\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}},\"traefik\":{\"Address\":\":8080\",\"TLS\":null,\"Redirect\":null,\"Auth\":null,\"WhitelistSourceRange\":null,\"WhiteList\":null,\"Compress\":false,\"ProxyProtocol\":null,\"ForwardedHeaders\":{\"Insecure\":true,\"TrustedIPs\":null}}},\"Cluster\":{\"Node\":\"c8656012-e164-4aff-ae78-766d3be83dd6\",\"Store\":{\"Store\":{},\"Prefix\":\"traefik\"}},\"Constraints\":[\"tag==traefik-public\"],\"ACME\":{\"Email\":\"mark@mobilea.nl\",\"Domains\":[{\"Main\":\"*.marktest.dev.mobilea.nl\",\"SANs\":null}],\"Storage\":\"acme.json\",\"StorageFile\":\"\",\"OnDemand\":false,\"OnHostRule\":true,\"CAServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"EntryPoint\":\"https\",\"KeyType\":\"\",\"DNSChallenge\":{\"Provider\":\"transip\",\"DelayBeforeCheck\":10000000000,\"Resolvers\":null,\"DisablePropagationCheck\":false},\"HTTPChallenge\":null,\"TLSChallenge\":null,\"DNSProvider\":\"\",\"DelayDontCheckDNS\":0,\"ACMELogging\":true,\"OverrideCertificates\":false,\"TLSConfig\":null},\"DefaultEntryPoints\":[\"http\"],\"ProvidersThrottleDuration\":2000000000,\"MaxIdleConnsPerHost\":200,\"IdleTimeout\":0,\"InsecureSkipVerify\":false,\"RootCAs\":null,\"Retry\":null,\"HealthCheck\":{\"Interval\":30000000000},\"RespondingTimeouts\":null,\"ForwardingTimeouts\":null,\"AllowMinWeightZero\":false,\"KeepTrailingSlash\":false,\"Web\":null,\"Docker\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":null,\"Trace\":false,\"TemplateVersion\":2,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"unix:///var/run/docker.sock\",\"Domain\":\"\",\"TLS\":null,\"ExposedByDefault\":false,\"UseBindPortIP\":false,\"SwarmMode\":true,\"Network\":\"\",\"SwarmModeRefreshSeconds\":15},\"File\":null,\"Marathon\":null,\"Consul\":{\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"consul-leader:8500\",\"Prefix\":\"traefik\",\"TLS\":null,\"Username\":\"\",\"Password\":\"\"},\"ConsulCatalog\":null,\"Etcd\":null,\"Zookeeper\":null,\"Boltdb\":null,\"Kubernetes\":null,\"Mesos\":null,\"Eureka\":null,\"ECS\":null,\"Rancher\":null,\"DynamoDB\":null,\"ServiceFabric\":null,\"Rest\":null,\"API\":{\"EntryPoint\":\"traefik\",\"Dashboard\":true,\"Debug\":false,\"CurrentConfigurations\":null,\"Statistics\":null},\"Metrics\":null,\"Ping\":null,\"HostResolver\":null}",
time="2019-07-23T09:36:40Z" level=info msg="Starting ACME renew job..."

@dduportal I also made the changes as described in https://github.com/tiangolo/dockerswarm.rocks/pull/22/files. But still the same output.

Are we sure the TXT record is created correctly on the domain?

The reproduction case is that I try to setup an environment with SSL enabled on internal hosts for each developer to run the environment on their own computer. We need SSL enabled since we develop a WebRTC application that needs SSL enabled locally to be able to test and develop.

@dduportal I just solved the problem. I also added the option:

--acme.dnsChallenge.resolvers="ns0.transip.net,ns1.transip.nl,ns2.transip.eu,1.1.1.1,8.8.8.8"

Which does create a wildcard certificate and adds it to the domain:

So setting the dns resolvers solved the issue for A records!

Also now I tried to do the same for CNAME records using the experimental features. This does not work (yet).

@dduportal Thanks for all the help and pointing me to some directions!

1 Like