"Unable to obtain ACME certificate for domains..."

Traefik Traefik v2
1

Not sure what to include here so I'll start simple seeing how I am a major novice when it comes to this kind of stuff (yml files intimidate me).

I had traefik working perfectly on my Unraid server. Than I had to do a reboot a few days ago an now I get this error.
time="2023-01-25T22:34:12-05:00" level=error msg="Unable to obtain ACME certificate for domains \"example.xyz,*.example.xyz\"" rule="Host(auth.example.xyz)" error="unable to generate a certificate for the domains [example.xyz *.example.xyz]: error: one or more domains had a problem:\n[*.example.xyz] time limit exceeded: last error: NS cruz.ns.cloudflare.com. returned REFUSED for _acme-challenge.example.xyz.\n[example.xyz] time limit exceeded: last error: NS cruz.ns.cloudflare.com. returned REFUSED for _acme-challenge.example.xyz.\n" providerName=letsencrypt.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=auth@file

I kind of get what it is saying but I have no idea how to fix it. I haven't changed my fileConfig.yml or traefik.yml. Any ideas what's going on? Please help.

2

@punted Thanks for your interest in Traefik!

Could you share your Traefik configuration files?

Did you check your domain settings, can you, for example, ping the domains?

3

Do you use Cloudflare just as DNS provider or have you enabled any security features and Cloudflare is proxying/tunneling all requests?

4

To answer your second question first. I can ping the domain.

To answer your first question see below:
Please let me know if I accidentally left in any sensitive info that should be removed. Thank you.

This is my fileConfig.yml code.

http:
  ## ROUTERS ##
  routers:
    organizrV2:
      entryPoints: 
       - https 
      rule: 'Host(`www.example.xyz`)'
      service: organizrV2 
      middlewares:
       - "securityHeaders"
       - "gzip"
       - "my-traefik-real-ip"
       - "my-plugin-log4shell"
       - "my-plugin-simplecache"
       - "my-geoblock"
    auth:
      entryPoints: 
       - https 
      rule: 'Host(`auth.example.xyz`)'
      service: auth 
      middlewares:
       - "securityHeaders"
       - "gzip"
       - "my-traefik-real-ip"
       - "my-plugin-log4shell"
       - "my-plugin-simplecache"
       - "my-geoblock"
    overseerr:
      entryPoints: 
       - https 
      rule: 'Host(`overseerr.example.xyz`)'
      service: overseerr 
      middlewares:
        - "securityHeaders"
        #- "auth"
        - "gzip"
        - "my-traefik-real-ip"
        - "my-plugin-log4shell"
        - "my-plugin-simplecache"
        - "my-geoblock"
    sonarr:
      entryPoints: 
       - https 
      rule: 'Host(`sonarr.example.xyz`)'
      service: sonarr 
      middlewares:
       - "securityHeaders"
       - "auth"
       - "gzip"
       - "my-traefik-real-ip"
       - "my-plugin-log4shell"
       - "my-plugin-simplecache"
       - "my-geoblock"
       - "sonarr-themepark"
    radarr:
      entryPoints: 
       - https 
      rule: 'Host(`radarr.example.xyz`)'
      service: radarr 
      middlewares:
       - "securityHeaders"
       - "auth"
       - "gzip"
       - "my-traefik-real-ip"
       - "my-plugin-log4shell"
       - "my-plugin-simplecache"
       - "my-geoblock"
       - "radarr-themepark"


  ## SERVICES ##
  services:
    organizrV2:
      loadbalancer:
        servers:
          - url: http://organizrv2:80
    auth:
      loadbalancer:
        servers:
          - url: http://authentik:9000
    overseerr:
      loadbalancer:
        servers:
          - url: http://overseerr:5055
    sonarr:
      loadbalancer:
        servers:
          - url: http://sonarr:8989
    radarr:
      loadbalancer:
        servers:
          - url: http://radarr:7878

 
  ## MIDDLEWARES ##
  middlewares:
    # Authentik guard
    auth:
      forwardauth:
        address: http://authentik:9000/outpost.goauthentik.io/auth/traefik
        trustForwardHeader: true
        authResponseHeaders:
          - X-authentik-username
          - X-authentik-groups
          - X-authentik-email
          - X-authentik-name
          - X-authentik-uid
          - X-authentik-jwt
          - X-authentik-meta-jwks
          - X-authentik-meta-outpost
          - X-authentik-meta-provider
          - X-authentik-meta-app
          - X-authentik-meta-version
    # Security headers
    securityHeaders:
      headers:
        customResponseHeaders:
          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex"
          X-Forwarded-Proto: "https"
          server: ""
        customRequestHeaders:
          X-Forwarded-Proto: "https"
        sslProxyHeaders:
          X-Forwarded-Proto: "https"
        referrerPolicy: "same-origin"
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        contentTypeNosniff: true
        browserXssFilter: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsSeconds: 63072000
        stsPreload: true
    #gzip
    gzip:
      compress: {}
    #Crowdsec stuff
    crowdsec-bouncer:
      forwardauth:
        address: http://crowdsec-traefik-bouncer:8080/api/v1/forwardAuth
        trustForwardHeader: true      

    #PLUGINS!!!
    my-traefik-real-ip:
        plugin:
            traefik-real-ip:
                excludednets:
                    - 1.1.1.1/24
    my-plugin-log4shell:
        plugin:
            plugin-log4shell:
                errorCode: "200"
    my-plugin-simplecache:
        plugin:
            plugin-simplecache:
                path: /tmp
    my-geoblock:
            plugin:
                geoblock:
                    allowLocalRequests: "false"
                    allowUnknownCountries: "false"
                    api: https://get.geojs.io/v1/ip/country/{ip}
                    apiTimeoutMs: "150"
                    cacheSize: "15"
                    countries:
                        - CH
                        - US
                        - SG
                    forceMonthlyUpdate: "true"
                    logAllowedRequests: "false"
                    logApiRequests: "true"
                    logLocalRequests: "false"
                    unknownCountryApiResponse: nil
    sonarr-themepark:
            plugin:
                traefik-themepark:
                    app: sonarr
                    theme: nord
    radarr-themepark:
            plugin:
                traefik-themepark:
                    app: radarr
                    theme: nord


# Only use secure ciphers - https://ssl-config.mozilla.org/#server=traefik&version=2.6.0&config=intermediate&guideline=5.6              
tls:
  options:
      default:
            minVersion: VersionTLS12
            cipherSuites:
              - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
              - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
              - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
              - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
              - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
              - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

.
And this is my traefik.yml code

global:
  checkNewVersion: true
  sendAnonymousUsage: false

serversTransport:
  insecureSkipVerify: true

#Crowdsec Stuff
accessLog:
  filePath: "/var/log/crowdsec/traefik.log"
  bufferingSize: 50

entryPoints:
  # Not used in apps, but redirect everything from HTTP to HTTPS
  http:
    address: :80
    forwardedHeaders:
      trustedIPs: &trustedIps
        # Start of Clouflare public IP list for HTTP requests, remove this if you don't use it
        - 172.18.0.0/16 #ADD YOUR DOCKER NETWORK HERE!!! 172.18.0.21/16
        - 173.245.48.0/20
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 141.101.64.0/18
        - 108.162.192.0/18
        - 190.93.240.0/20
        - 188.114.96.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 162.158.0.0/15
        - 104.16.0.0/13
        - 104.24.0.0/14
        - 172.64.0.0/13
        - 131.0.72.0/22
        - 2400:cb00::/32
        - 2606:4700::/32
        - 2803:f800::/32
        - 2405:b500::/32
        - 2405:8100::/32
        - 2a06:98c0::/29
        - 2c0f:f248::/32
        # End of Cloudlare public IP list
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https

  # HTTPS endpoint, with domain wildcard
  https:
    address: :443
    forwardedHeaders:
      # Reuse list of Cloudflare Trusted IP's above for HTTPS requests
      trustedIPs: *trustedIps
    http:
      tls:
        # Generate a wildcard domain certificate
        certResolver: letsencrypt
        domains:
          - main: example.xyz
            sans:
              - '*.example.xyz'
      # middlewares:
      #   - securityHeaders@file
      #   - my-traefik-real-ip@file
      #   - crowdsec-bouncer@file
      #   - gzip@file
      #   - my-plugin-log4shell@file
      #   - my-plugin-simplecache@file
      #   - my-traefik-themepark@file
      #   - my-geoblock@file


providers:
  providersThrottleDuration: 2s

  # File provider for connecting things that are outside of docker / defining middleware
  file:
    filename: /etc/traefik/fileConfig.yml
    watch: true

  # Docker provider for connecting all apps that are inside of the docker network
  docker:
    watch: true
    network: example-net    # Add Your Docker Network Name Here
    # Default host rule to containername.domain.example
    defaultRule: "Host(`{{ lower (trimPrefix `/` .Name )}}.example.xyz`)"    # Replace with your domain
    swarmModeRefreshSeconds: 15s
    exposedByDefault: false
    endpoint: "tcp://dockersocket:2375" # Uncomment if you are using docker socket proxy


# Enable traefik ui
api:
  dashboard: true
  insecure: true


# Log level INFO|DEBUG|ERROR
log:
  level: INFO


# Use letsencrypt to generate ssl serficiates
certificatesResolvers:
  letsencrypt:
    acme:
      email: admin@example.xyz
      storage: /etc/traefik/acme.json
      dnsChallenge:
        provider: cloudflare
        # Used to make sure the dns challenge is propagated to the rights dns servers
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

experimental:
  plugins:
    traefik-real-ip:
      moduleName: "github.com/soulbalz/traefik-real-ip"
      version: "v1.0.3"

    plugin-log4shell:
      moduleName: "github.com/traefik/plugin-log4shell"
      version: "v0.1.2"

    plugin-simplecache:
      moduleName: "github.com/traefik/plugin-simplecache"
      version: "v0.2.1"

    traefik-themepark:
      moduleName: "github.com/packruler/traefik-themepark"
      version: "v1.2.2"

    geoblock:
      moduleName: "github.com/PascalMinder/geoblock"
      version: "v0.2.3"
5

I believe I am using Cloudflares security features. I followed the ibracorp guides to set up my unraid server. One of the trouble shooting steps I took was to redo the guide again but that did not fix my problem.