Need Help! Cloudflare: failed to find zone DOMAIN.net

Hello! I've spent a couple of hours messing with this trying to get it working but I cant seem to figure out what I'm doing wrong. I've been trying to get ssl certificates for apps on my local ip, but I can not correctly pull the certificates. I've tried messing with my Cloudflare, I've messed with the configs of my pi-hole instance and traefik, but nothing has come of it yet. If anybody can help me I'd really appreciate it!

These are my configs, and logs, I can also provide anything else needed.

docker-compose.yml

version: '3'

services:
  traefik:
    image: "traefik:latest"
    container_name: "traefik"
    security_opt:
      - no-new-privileges:true
    networks:
      - proxy
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - CLOUDFLARE_DNS_API_TOKEN="..."
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /mnt/nfs/appdata/traefik/data/traefik.yml:/traefik.yml:ro
      - /mnt/nfs/appdata/traefik/data/acme.json:/acme.json
      - /mnt/nfs/appdata/traefik/data/config.yml:/config.yml:ro

    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.local.DOMAIN.net`)"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.local.DOMAIN.net`)"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.tls.domains[0].main=local.DOMAIN.net"
      - "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.local.DOMAIN.net"
      - "traefik.http.routers.traefik-secure.service=api@internal"

networks:
  proxy:
    external: true

traefik.yml:

api:
  dashboard: true
  debug: true
  insecure: true
entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https
  https:
    address: ":443"
log:
  level: DEBUG
serversTransport:
  insecureSkipVerify: true
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: /config.yml
certificatesResolvers:
  cloudflare:
    acme:
      email: email@gmail.com
      storage: acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

logs:

time="2023-03-26T07:17:22Z" level=info msg="Configuration loaded from file: /traefik.yml"
time="2023-03-26T07:17:22Z" level=info msg="Traefik version 2.9.9 built on 2023-03-21T15:52:28Z"
time="2023-03-26T07:17:22Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"insecureSkipVerify\":true,\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"https\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"https\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"watch\":true,\"filename\":\"/config.yml\"}},\"api\":{\"insecure\":true,\"dashboard\":true,\"debug\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"cloudflare\":{\"acme\":{\"email\":\"email.com@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"cloudflare\",\"resolvers\":[\"1.1.1.1:53\",\"1.0.0.1:53\"]}}}}}"
time="2023-03-26T07:17:22Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
time="2023-03-26T07:17:22Z" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
time="2023-03-26T07:17:22Z" level=debug msg="Starting TCP Server" entryPointName=https
time="2023-03-26T07:17:22Z" level=debug msg="Starting TCP Server" entryPointName=traefik
time="2023-03-26T07:17:22Z" level=debug msg="Starting TCP Server" entryPointName=http
time="2023-03-26T07:17:22Z" level=info msg="Starting provider *file.Provider"
time="2023-03-26T07:17:22Z" level=debug msg="*file.Provider provider configuration: {\"watch\":true,\"filename\":\"/config.yml\"}"
time="2023-03-26T07:17:22Z" level=info msg="Starting provider *traefik.Provider"
time="2023-03-26T07:17:22Z" level=debug msg="*traefik.Provider provider configuration: {}"
time="2023-03-26T07:17:22Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect@internal\",\"dashboard_stripprefix@internal\"],\"service\":\"dashboard@internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645},\"debug\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/debug`)\",\"priority\":2147483646},\"http-to-https\":{\"entryPoints\":[\"http\"],\"middlewares\":[\"redirect-http-to-https\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}},\"redirect-http-to-https\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"serversTransports\":{\"default\":{\"insecureSkipVerify\":true,\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-03-26T07:17:22Z" level=info msg="Starting provider *docker.Provider"
time="2023-03-26T07:17:22Z" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"}"
time="2023-03-26T07:17:22Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-03-26T07:17:22Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-03-26T07:17:22Z" level=info msg="Starting provider *acme.Provider"
time="2023-03-26T07:17:22Z" level=debug msg="*acme.Provider provider configuration: {\"email\":\"email.com@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"cloudflare\",\"resolvers\":[\"1.1.1.1:53\",\"1.0.0.1:53\"]},\"ResolverName\":\"cloudflare\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
time="2023-03-26T07:17:22Z" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-03-26T07:17:22Z" level=info msg="Testing certificate renew..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=file
time="2023-03-26T07:17:22Z" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="Provider connection established with docker 23.0.1 (API 1.42)" providerName=docker
time="2023-03-26T07:17:22Z" level=debug msg="Filtering disabled container" providerName=docker container=pihole-pihole-55d9063bb15badd6b768ac361dba65ba954185d2f50bac0263d216db091a8746
time="2023-03-26T07:17:22Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"traefik\":{\"entryPoints\":[\"http\"],\"middlewares\":[\"traefik-https-redirect\"],\"service\":\"traefik-traefik\",\"rule\":\"Host(`traefik-dashboard.local.DOMAIN.net`)\"},\"traefik-secure\":{\"entryPoints\":[\"https\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik-dashboard.local.DOMAIN.net`)\",\"tls\":{\"certResolver\":\"cloudflare\",\"domains\":[{\"main\":\"local.DOMAIN.net\",\"sans\":[\"*.local.DOMAIN.net\"]}]}}},\"services\":{\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.18.0.3:80\"}],\"passHostHeader\":true}}},\"middlewares\":{\"sslheader\":{\"headers\":{\"customRequestHeaders\":{\"X-Forwarded-Proto\":\"https\"}}},\"traefik-https-redirect\":{\"redirectScheme\":{\"scheme\":\"https\"}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-03-26T07:17:22Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=api@internal
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" routerName=dashboard@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" routerName=dashboard@internal entryPointName=traefik middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix
time="2023-03-26T07:17:22Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal entryPointName=traefik
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-03-26T07:17:22Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-03-26T07:17:22Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_redirect@internal entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=debug@internal middlewareName=tracing
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware noop@internal" middlewareType=TracingForwarder middlewareName=tracing entryPointName=http routerName=http-to-https@internal
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" middlewareType=RedirectScheme entryPointName=http routerName=http-to-https@internal middlewareName=redirect-http-to-https@internal
time="2023-03-26T07:17:22Z" level=debug msg="Setting up redirection to https 443" middlewareType=RedirectScheme entryPointName=http routerName=http-to-https@internal middlewareName=redirect-http-to-https@internal
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=http
time="2023-03-26T07:17:22Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=api@internal middlewareName=tracing
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix entryPointName=traefik
time="2023-03-26T07:17:22Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal entryPointName=traefik
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T07:17:22Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T07:17:22Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=debug@internal
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" middlewareName=pipelining entryPointName=http routerName=traefik@docker serviceName=traefik-traefik middlewareType=Pipelining
time="2023-03-26T07:17:22Z" level=debug msg="Creating load-balancer" routerName=traefik@docker serviceName=traefik-traefik entryPointName=http
time="2023-03-26T07:17:22Z" level=debug msg="Creating server 0 http://172.18.0.3:80" serverName=0 routerName=traefik@docker serviceName=traefik-traefik entryPointName=http
time="2023-03-26T07:17:22Z" level=debug msg="child http://172.18.0.3:80 now UP"
time="2023-03-26T07:17:22Z" level=debug msg="Propagating new UP status"
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware traefik-traefik" entryPointName=http routerName=traefik@docker middlewareName=tracing middlewareType=TracingForwarder
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=http routerName=traefik@docker middlewareName=traefik-https-redirect@docker middlewareType=RedirectScheme
time="2023-03-26T07:17:22Z" level=debug msg="Setting up redirection to https " middlewareName=traefik-https-redirect@docker middlewareType=RedirectScheme entryPointName=http routerName=traefik@docker
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware noop@internal" middlewareType=TracingForwarder entryPointName=http routerName=http-to-https@internal middlewareName=tracing
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" middlewareType=RedirectScheme middlewareName=redirect-http-to-https@internal entryPointName=http routerName=http-to-https@internal
time="2023-03-26T07:17:22Z" level=debug msg="Setting up redirection to https 443" middlewareType=RedirectScheme middlewareName=redirect-http-to-https@internal entryPointName=http routerName=http-to-https@internal
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-03-26T07:17:22Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing entryPointName=https routerName=traefik-secure@docker middlewareType=TracingForwarder
time="2023-03-26T07:17:22Z" level=debug msg="Creating middleware" entryPointName=https middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-03-26T07:17:22Z" level=debug msg="Adding route for traefik-dashboard.local.DOMAIN.net with TLS options default" entryPointName=https
time="2023-03-26T07:17:22Z" level=debug msg="Looking for provided certificate(s) to validate [\"local.DOMAIN.net\" \"*.local.DOMAIN.net\"]..." providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-03-26T07:17:22Z" level=debug msg="Domains [\"local.DOMAIN.net\" \"*.local.DOMAIN.net\"] need ACME certificates generation for domains \"local.DOMAIN.net,*.local.DOMAIN.net\"." providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-03-26T07:17:22Z" level=debug msg="Loading ACME certificates [local.DOMAIN.net *.local.DOMAIN.net]..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="Building ACME client..." providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="Using DNS Challenge provider: cloudflare" providerName=cloudflare.acme
time="2023-03-26T07:17:22Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net, *.local.DOMAIN.net] acme: Obtaining bundled SAN certificate"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [*.local.DOMAIN.net] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214098325147"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214098325157"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [*.local.DOMAIN.net] acme: use dns-01 solver"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] acme: Could not find solver for: tls-alpn-01"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] acme: Could not find solver for: http-01"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] acme: use dns-01 solver"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [*.local.DOMAIN.net] acme: Preparing to solve DNS-01"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] Found CNAME entry for \"_acme-challenge.local.DOMAIN.net.\": \"local.DOMAIN.net.\""
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] acme: Preparing to solve DNS-01"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] Found CNAME entry for \"_acme-challenge.local.DOMAIN.net.\": \"local.DOMAIN.net.\""
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] [*.local.DOMAIN.net] acme: Cleaning DNS-01 challenge"
time="2023-03-26T07:17:23Z" level=debug msg="legolog: [INFO] Found CNAME entry for \"_acme-challenge.local.DOMAIN.net.\": \"local.DOMAIN.net.\""
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [WARN] [*.local.DOMAIN.net] acme: cleaning up failed: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003) "
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [INFO] [local.DOMAIN.net] acme: Cleaning DNS-01 challenge"
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [INFO] Found CNAME entry for \"_acme-challenge.local.DOMAIN.net.\": \"local.DOMAIN.net.\""
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [WARN] [local.DOMAIN.net] acme: cleaning up failed: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003) "
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214098325147"
time="2023-03-26T07:17:24Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214098325157"
time="2023-03-26T07:17:24Z" level=error msg="Unable to obtain ACME certificate for domains \"local.DOMAIN.net,*.local.DOMAIN.net\"" error="unable to generate a certificate for the domains [local.DOMAIN.net *.local.DOMAIN.net]: error: one or more domains had a problem:\n[*.local.DOMAIN.net] [*.local.DOMAIN.net] acme: error presenting token: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003)\n[local.DOMAIN.net] [local.DOMAIN.net] acme: error presenting token: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003)\n" routerName=traefik-secure@docker rule="Host(`traefik-dashboard.local.DOMAIN.net`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T07:17:25Z" level=debug msg="Serving default certificate for request: \"traefik-dashboard.local.DOMAIN.net\""
time="2023-03-26T07:17:26Z" level=debug msg="http: TLS handshake error from 192.168.1.9:65108: remote error: tls: bad certificate"

heres my cloudflare api setup too:

Thank you in advance!

How is Cloudflare configured? Is it just your DNS? Is it proxying all requests, does it forward port 80+443?

BTW: you have https redirect in your static config (traefik.yml), no need for it in labels again.

Have you tried to use one level less, like traefik-dashboard.DOMAIN.net with according main/sans?

How is Cloudflare configured? Is it just your DNS?

This was my cloudflare dns setup, I was trying to use pihole at first for the DNS but was also getting the same zone error.

Is it proxying all requests, does it forward port 80+443?

I think so, it does correctly attempt to put an ssl certificate over "traefik.DOMAIN.net" its just the traefik default certificate.

Also I updated my docker-compose like you recommended to:

version: '3'

services:
  traefik:
    image: "traefik:latest"
    container_name: "traefik"
    security_opt:
      - no-new-privileges:true
    networks:
      - proxy
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - CLOUDFLARE_DNS_API_TOKEN="..."
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /mnt/nfs/appdata/traefik/data/traefik.yml:/traefik.yml:ro
      - /mnt/nfs/appdata/traefik/data/acme.json:/acme.json
      - /mnt/nfs/appdata/traefik/data/config.yml:/config.yml:ro
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.DOMAIN.net`)"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik-secure.service=api@internal"
networks:
  proxy:
    external: true

but I still get the same error:

time="2023-03-26T20:20:35Z" level=info msg="Configuration loaded from file: /traefik.yml"
time="2023-03-26T20:20:35Z" level=info msg="Traefik version 2.9.9 built on 2023-03-21T15:52:28Z"
time="2023-03-26T20:20:35Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"insecureSkipVerify\":true,\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{\"redirections\":{\"entryPoint\":{\"to\":\"https\",\"scheme\":\"https\",\"permanent\":true,\"priority\":2147483646}}},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"https\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"watch\":true,\"filename\":\"/config.yml\"}},\"api\":{\"insecure\":true,\"dashboard\":true,\"debug\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"cloudflare\":{\"acme\":{\"email\":\"email@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"cloudflare\",\"resolvers\":[\"1.1.1.1:53\",\"1.0.0.1:53\"]}}}}}"
time="2023-03-26T20:20:35Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
time="2023-03-26T20:20:35Z" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
time="2023-03-26T20:20:35Z" level=debug msg="Starting TCP Server" entryPointName=https
time="2023-03-26T20:20:35Z" level=debug msg="Starting TCP Server" entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Starting TCP Server" entryPointName=http
time="2023-03-26T20:20:35Z" level=info msg="Starting provider *file.Provider"
time="2023-03-26T20:20:35Z" level=debug msg="*file.Provider provider configuration: {\"watch\":true,\"filename\":\"/config.yml\"}"
time="2023-03-26T20:20:35Z" level=info msg="Starting provider *traefik.Provider"
time="2023-03-26T20:20:35Z" level=debug msg="*traefik.Provider provider configuration: {}"
time="2023-03-26T20:20:35Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect@internal\",\"dashboard_stripprefix@internal\"],\"service\":\"dashboard@internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645},\"debug\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/debug`)\",\"priority\":2147483646},\"http-to-https\":{\"entryPoints\":[\"http\"],\"middlewares\":[\"redirect-http-to-https\"],\"service\":\"noop@internal\",\"rule\":\"HostRegexp(`{host:.+}`)\",\"priority\":2147483646}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}},\"redirect-http-to-https\":{\"redirectScheme\":{\"scheme\":\"https\",\"port\":\"443\",\"permanent\":true}}},\"serversTransports\":{\"default\":{\"insecureSkipVerify\":true,\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-03-26T20:20:35Z" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=file
time="2023-03-26T20:20:35Z" level=info msg="Starting provider *docker.Provider"
time="2023-03-26T20:20:35Z" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmModeRefreshSeconds\":\"15s\"}"
time="2023-03-26T20:20:35Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-03-26T20:20:35Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-03-26T20:20:35Z" level=info msg="Starting provider *acme.Provider"
time="2023-03-26T20:20:35Z" level=debug msg="*acme.Provider provider configuration: {\"email\":\"email@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"dnsChallenge\":{\"provider\":\"cloudflare\",\"resolvers\":[\"1.1.1.1:53\",\"1.0.0.1:53\"]},\"ResolverName\":\"cloudflare\",\"store\":{},\"TLSChallengeProvider\":{},\"HTTPChallengeProvider\":{}}"
time="2023-03-26T20:20:35Z" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T20:20:35Z" level=info msg="Testing certificate renew..." providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
time="2023-03-26T20:20:35Z" level=debug msg="Configuration received: {\"http\":{},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=cloudflare.acme
time="2023-03-26T20:20:35Z" level=debug msg="Provider connection established with docker 23.0.1 (API 1.42)" providerName=docker
time="2023-03-26T20:20:35Z" level=debug msg="Filtering disabled container" providerName=docker container=pihole-pihole-55d9063bb15badd6b768ac361dba65ba954185d2f50bac0263d216db091a8746
time="2023-03-26T20:20:35Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"traefik-secure\":{\"entryPoints\":[\"https\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.DOMAIN.net`)\",\"tls\":{\"certResolver\":\"cloudflare\"}}},\"services\":{\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.18.0.3:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-03-26T20:20:35Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware noop@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=http routerName=http-to-https@internal
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme entryPointName=http routerName=http-to-https@internal
time="2023-03-26T20:20:35Z" level=debug msg="Setting up redirection to https 443" entryPointName=http routerName=http-to-https@internal middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=http
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" entryPointName=traefik routerName=dashboard@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix
time="2023-03-26T20:20:35Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik middlewareName=dashboard_stripprefix@internal routerName=dashboard@internal
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-03-26T20:20:35Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_redirect@internal entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware api@internal" entryPointName=traefik routerName=debug@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware api@internal" entryPointName=traefik routerName=api@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" middlewareType=Recovery middlewareName=traefik-internal-recovery entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware api@internal" routerName=debug@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware api@internal" routerName=api@internal middlewareType=TracingForwarder middlewareName=tracing entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal middlewareName=tracing
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix entryPointName=traefik
time="2023-03-26T20:20:35Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_stripprefix@internal entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-03-26T20:20:35Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-03-26T20:20:35Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_redirect@internal entryPointName=traefik routerName=dashboard@internal
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery entryPointName=traefik middlewareType=Recovery
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware noop@internal" entryPointName=http routerName=http-to-https@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" entryPointName=http routerName=http-to-https@internal middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme
time="2023-03-26T20:20:35Z" level=debug msg="Setting up redirection to https 443" entryPointName=http routerName=http-to-https@internal middlewareName=redirect-http-to-https@internal middlewareType=RedirectScheme
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=http middlewareName=traefik-internal-recovery
time="2023-03-26T20:20:35Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder routerName=traefik-secure@docker entryPointName=https middlewareName=tracing
time="2023-03-26T20:20:35Z" level=debug msg="Creating middleware" entryPointName=https middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-03-26T20:20:35Z" level=debug msg="Adding route for traefik.DOMAIN.net with TLS options default" entryPointName=https
time="2023-03-26T20:20:35Z" level=debug msg="Trying to challenge certificate for domain [traefik.DOMAIN.net] found in HostSNI rule" providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker rule="Host(`traefik.DOMAIN.net`)"
time="2023-03-26T20:20:35Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.DOMAIN.net\"]..." rule="Host(`traefik.DOMAIN.net`)" providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker
time="2023-03-26T20:20:35Z" level=debug msg="Domains [\"traefik.DOMAIN.net\"] need ACME certificates generation for domains \"traefik.DOMAIN.net\"." providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker rule="Host(`traefik.DOMAIN.net`)"
time="2023-03-26T20:20:35Z" level=debug msg="Loading ACME certificates [traefik.DOMAIN.net]..." providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker rule="Host(`traefik.DOMAIN.net`)"
time="2023-03-26T20:20:35Z" level=debug msg="Building ACME client..." providerName=cloudflare.acme
time="2023-03-26T20:20:35Z" level=debug msg="https://acme-v02.api.letsencrypt.org/directory" providerName=cloudflare.acme
time="2023-03-26T20:20:36Z" level=debug msg="Using DNS Challenge provider: cloudflare" providerName=cloudflare.acme
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: Obtaining bundled SAN certificate"
time="2023-03-26T20:20:36Z" level=debug msg="Serving default certificate for request: \"traefik-dashboard.local.DOMAIN.net\""
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214237820737"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: Could not find solver for: tls-alpn-01"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: Could not find solver for: http-01"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: use dns-01 solver"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: Preparing to solve DNS-01"
time="2023-03-26T20:20:36Z" level=debug msg="http: TLS handshake error from 192.168.1.9:51868: remote error: tls: bad certificate"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] [traefik.DOMAIN.net] acme: Cleaning DNS-01 challenge"
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [WARN] [traefik.DOMAIN.net] acme: cleaning up failed: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003) "
time="2023-03-26T20:20:36Z" level=debug msg="legolog: [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/214237820737"
time="2023-03-26T20:20:36Z" level=error msg="Unable to obtain ACME certificate for domains \"traefik.DOMAIN.net\": unable to generate a certificate for the domains [traefik.DOMAIN.net]: error: one or more domains had a problem:\n[traefik.DOMAIN.net] [traefik.DOMAIN.net] acme: error presenting token: cloudflare: failed to find zone DOMAIN.net.: ListZonesContext command failed: Invalid request headers (6003)\n" providerName=cloudflare.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=traefik-secure@docker rule="Host(`traefik.DOMAIN.net`)"
time="2023-03-26T20:20:41Z" level=debug msg="Serving default certificate for request: \"traefik-dashboard.local.DOMAIN.net\""
time="2023-03-26T20:20:41Z" level=debug msg="http: TLS handshake error from 192.168.1.9:51869: remote error: tls: bad certificate"
time="2023-03-26T20:20:46Z" level=debug msg="Serving default certificate for request: \"traefik-dashboard.local.DOMAIN.net\""
time="2023-03-26T20:20:46Z" level=debug msg="http: TLS handshake error from 192.168.1.9:51870: remote error: tls: bad certificate"

To debug step by step: For dnsChallenge there is a delayBeforeCheck parameter (docs). Set that to a couple of minutes, start Traefik and check manually on Cloudflare if the validation DNS entry is set.

By default, the provider verifies the TXT record before letting ACME verify. You can delay this operation by specifying a delay (in seconds) with delayBeforeCheck (value must be greater than zero).

:confused: I had quotes around my api key in the docker compose file...

Instead I should have had it like

      - CLOUDFLARE_DNS_API_TOKEN=...

After I got rid of the quotes, it started working.

UGHHHHH.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.