Wildcard Certificates from Let's Encrypt with Cloudlflare: Cannot negotiate ALPN protocol acme-tls/1 for tls-alpn-01 challenge

I want to get a wildcard cert from let's encrypt but I also use cloudflare. I have setup my traefik and services as defined by the docker-compose below. While the certificate works for the main domain (example.com), it does not seem to work for the subdomain (analytics.example.com). In traefik I get this log:

Unable to obtain ACME certificate for domains \"example.com,*.example.com\" : 
unable to generate a certificate for the domains [example.com *.example.com]: 
error: one or more domains had a problem:\n[example.com] acme: error: 403 :: 
urn:ietf:params:acme:error:unauthorized :: 
Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge\n" 
providerName=le.acme

Any idea what the probelm might be?

services:
  traefik:
    image: traefik:v2.8.3
    command:
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.web.http.redirections.entrypoint.scheme=https
      - --certificatesresolvers.le.acme.tlschallenge=true
      - --certificatesresolvers.le.acme.email=${EMAIL}
      - --certificatesresolvers.le.acme.dnschallenge=true
      - --certificatesresolvers.le.acme.dnschallenge.provider=cloudflare
      - --certificatesresolvers.le.acme.storage=/certs/acme.json
      - --certificatesresolvers.le.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      - --entrypoints.websecure.http.tls=true
      - --entrypoints.websecure.http.tls.certResolver=le
      - --entrypoints.websecure.http.tls.domains[0].main=example.com
      - --entrypoints.websecure.http.tls.domains[0].sans=*.example.com
    labels:
      - traefik.enable=true
      - traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)
      - traefik.http.routers.http-catchall.entrypoints=web
      - traefik.http.routers.http-catchall.middlewares=https-redirect
      - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
    environment:
      - CF_API_EMAIL=${EMAIL}
      - CF_API_KEY=${CLOUDFLARE_API_TOKEN}
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - certs:/certs
  frontend:
    #...
    labels:
      - traefik.enable=true
      - traefik.http.routers.frontend.priority=1
      - traefik.http.routers.frontend.entrypoints=websecure
      - traefik.http.routers.frontend.tls=true
      - traefik.http.routers.frontend.tls.certresolver=le
      - traefik.http.routers.frontend.rule=Host(`example.com`)

  analytics-tool:
    # ...
    labels:
      - traefik.enable=true
      - traefik.http.routers.analytics.entrypoints=websecure
      - traefik.http.routers.analytics.tls=true
      - traefik.http.routers.analytics.tls.certresolver=le
      - traefik.http.routers.analytics.rule=Host(`analytics.example.com`)

I had the same problem and I had a wrong DNS entry.

I believe you can not activate two challenge types on your le resolver at the same time.

We normally just use tlschallenge because that way you don't need to mess with your DNS provider. You only need dnschallenge if you really want to use wildcards. As long as every service has its own .rule=Host( xyz.example.com ) (you can use multiple hosts with ||), Traefik will just create an individual LE certificate for each service.

Not sure if you really need a http-catchall for https-redirect. You have a http->https redirect on entrypoint and websecure is set to TLS with .websecure.http.tls=true.