ACME DNS Challenge issues

 dig @108.162.192.227 SOA ragenetwork.me

; <<>> DiG 9.18.1-1ubuntu1.3-Ubuntu <<>> @108.162.192.227 SOA ragenetwork.me
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61150
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;ragenetwork.me.                        IN      SOA

;; ANSWER SECTION:
ragenetwork.me.         3600    IN      SOA     chance.ns.cloudflare.com. dns.cloudflare.com. 2304183249 10000 2400 604800 3600

;; Query time: 99 msec
;; SERVER: 108.162.192.227#53(108.162.192.227) (UDP)
;; WHEN: Mon Mar 13 19:36:52 AWST 2023
;; MSG SIZE  rcvd: 107

so maybe you are blocking UDP?

I dont think I am. I have my firewall rule blocking external DNS queries turned off, and I can do nslookups and dig queries etc :frowning:

hey, its been some time, have you managed to resolve this?

I've been fighting with this same problem for the last 4 days, running a combination of Opnsense with AdGuard Home running as an internal DNS server via Opnsense plugin, and Traefik on a separate machine that hosts all my docker stuff. I had two problems keeping this from working even though the results from dig looked correct best I could tell.

First problem was a misconfiguration of my NAT rule for DNS redirect, I forget exactly what I had toggled incorrectly but I caught the error when comparing my config to the documentation and some tutorials. There are lots of guides online but this one was the best I found as far as readability goes when comparing firewall rules.

Second problem was with my internal DNS, I had set a wildcard redirect set in AdGuard to redirect DNS requests for *.internal.my.domain to the IP address of my Docker host where Traefik is running. This would result in Traefik getting redirected to itself during a checking for the propagation of the _acme-challenge.internal.my.domain TXT record. Solution here was abandoning the wildcard redirect and setting specific DNS records for each service that was going to be on that domain.

I cant thank you enough, i though i was the only idiot in the world who has that problem and on top of that cant resolve it! Thanks!
My solution was just to remove wildcards from adguard home and let cloudflare handle redirects to my private IP address. Ill probably put them back on and keep removing them shortly before certificate expirations so my request dont even get to cloudflare unless needed, but at least now i know what to tweak!

One other question for you buddy if possible!
I checked that guide you sent and im wondering how did you set it up to properly work with adguard that is run directly on opnsense?
I see some guy commented in the guide to just replace PiHole address to 127.0.0.1 but how can that work when the source is not 127.0.0.1 on a given interface? I think he might be wrong as 127.0.0.1 is more of a destination than the source if im not mistaken. Maybe that Public-DNS list should be added straight into AdGuard block list or something? If i do it exactly how he did it (since im running another instance of adguard on my LAN interface) i see that that rule also blocks all of DNS requests to OPNsense itself(which should then be redirected to adguard home plugin), it basically disables my adguard home plugin and leaves only the secondary adguard working if you see what i mean? Thank you man so much!

If you don't mind setting up a unique DNS record for each service you're trying to proxy you won't have mess with removing it for cert updates. My DNS records in AdGuard look something like this now.

  • service1.internal.my.domain -> ip of my traefik host
  • service2.internal.my.domain -> ip of my traefik host
  • service3.internal.my.domain -> ip of my traefik host
  • etc...

I'm not sure which comment you're looking at but 127.0.0.1 is localhost, so I'm assuming they have a setup where any requests coming into the firewall for DNS (port 53) get redirected to port 53 on their OPNsense machine. That'd be similar to my setup except I'm using the actual network address of my OPNsense machine instead of the localhost address. I also have unbounDNS turned off and I'm running AdGuard on port 53 of my OPNsense machine. AdGuard has the following setup as upstream DNS servers:

Hopefully that gives you something to work with, I'm definitely an amateur but I could provide a little more guidance if you share your actual config assuming this doesn't help.

Yea that sucks because i have ~50 services....but youre saying basically every service except traefik.example.org?

I was talking about this comment

you said "That'd be similar to my setup except I'm using the actual network address of my OPNsense machine instead of the localhost address." What exactly do you mean by your OPNsense machine instead? Im assuming your setup would be like this one that i have found some time ago and reapplied it now in order to get these redirection working properly.


I also have unbound off and and only mullvad DNS as upstreams in my adguard, basically identical to yours.

I do have another question for you bud, could you please tell me how exactly did you setup the outbound NAT Rule 3 from the guide? Im not sure what to put under these sections
Source - Network for the outbound NAT mapping: Your internal LAN network
Destination - Type: Network
Destination - Network for the outbound NAT Mappings: Your PiHole’s IP Address

there is no network type in opnsense settings and im not sure what would be internal LAN network here, is it the same rule applied to each interface and VLAN with their internal networks respectively? and again, if it is outbound im assuimg instead of PiHole IP address i would put 127.0.0.1?
Thanks man even if you dont find the time to answer!

I use the actual network address of my OPNsense in my firewall rules, I couldn't get get it to work correctly using the localhost address. Since I'm running AdGuard via OPNsense plugin instead of PiHole on a separate machine the IP address of the DNS server is just the same as the IP address of the OPNsense machine and anything coming into OPNsense on port 53 for DNS goes to AdGuard....so if my OPNsense firewall is on 192.168.1.1 my rules would look like this.

  1. Firewall -> NAT -> Port Forward
  • Interface - LAN
  • Proto - TCP/UDP
  • Souce Addr - *
  • Source Port - *
  • Dest Addr - !LAN net
  • Dest Port - 53 (DNS)
  • NAT IP - 192.168.1.1
  • NAT Port - 53 (DNS)
  1. Firewall -> NAT -> Outbound
  • Interface - LAN
  • Souce - LAN net
  • Source Port - *
  • Dest - 192.168.1.1
  • Dest Port - 53
  • NAT Addr - Interface addr
  • NAT Port - *
  • Static Port - NO

UPDATE...

I think I got it working with the wildcard DNS rewrite in AdGuard. You can set exceptions to rewrite rules in AdGuard by rewriting the DNS record to itself (see here). If you have a wildcard rewrite of
*.my.domain -> your Traefik host IP then a second rewrite of
_acme-challenge.my.domain -> _acme-challenge.my.domain will bypass the wildcard and allow the propagation checks to work.

1 Like

So you are saying that if i have two rewrites, like this example:

*.Google.com -- 10.0.0.1
Test.Google.com -- google.com

When i go to test.google.com it will actually take me tk google's website? Im not sure if i understood correctly but i believe that is the way of making an exception for woldcard?

What I'm saying is if you use two rewrites like this the wildcard will direct everything for my.domain to 192.168.1.20 except _acme-challenge.my.domain which will just use your normal DNS settings. This allows Traefik to do the DNS propagation check when it attempts to renew certificates.

I encounter the same issue. I use OpenWRT and OpenClash (fake IP mode with Clash meta core), in conjunction with AdGuard Home for ad blocking. All of these are hosted on a separate machine, which functions as my homelab router. The traffic flow is from dnsmasq to AdGuard Home, then to OpenClash, and finally to the extranet.

On another internal machine(window with wsl), I can obtain certificates using acme.sh, but with Traefik's Lego, I'm unable to do so.

After some searching, I've configured my setup to use an acme.sh container to obtain certificates and then mount them to Traefik.
Another solution involves setting a DNS resolver for the Traefik container, like this:

docker-compose.yml

services:
  traefik:
    image: traefik:v3.0
    restart: always
    dns:
      - 1.1.1.1
      - 1.0.0.1
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./.traefik/conf:/etc/traefik
      - ./.traefik/logs:/var/logs
      - ./.acme.sh:/.acme.sh
  ...
  acme.sh:
    image: neilpang/acme.sh
    volumes:
      - ./.acme.sh:/acme.sh
    command: daemon

traefik.yaml

certificatesResolvers:
  duckdns:
    acme:
      email: <YOUr_EMAIL>
      storage: /etc/traefik/cert/acme.json
      caServer: https://acme-v02.api.letsencrypt.org/directory
      dnsChallenge:
        provider: duckdns
        delayBeforeCheck: "5"
        # resolvers:
        #   - "1.1.1.1:53"
        #   - "1.0.0.1:53"

  duckdns-staging:
    acme:
      email: <YOUr_EMAIL>
      storage: /etc/traefik/cert/acme-staging.json
      caServer: https://acme-staging-v02.api.letsencrypt.org/directory
      dnsChallenge:
        provider: duckdns
        delayBeforeCheck: "5"
        # resolvers:
        #   - "1.1.1.1:53"
        #   - "1.0.0.1:53"

I'm confident that there might be some misconfiguration with the OpenWRT router, but for now, this setup is working for me.