Sub-level subdomains not working

When using domains like ..mydomain.com most of the times traefik in my config just refuses to proxy I would do a few restarts it might work and then go back to being broken. It will also not issue any wild card certs again few restarts it might work but most of the times it just doesnt cooperate. One level deep subdomains however work perfectly fine with no issues.

My traefik.yml (I have also tried adding additional wildcard domains in sans which I know isn't required but that also seems to help only intermittently)

global:
  checkNewVersion: true

serversTransport:
  insecureSkipVerify: true

entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https

  https:
    address: ":443"
    http:
      tls:
        certResolver: letsencrypt
        domains:
          - main: "{{ domain_name }}"
            sans:
              - "*.{{ domain_name }}"
      middlewares:
        - securityHeaders@file

providers:
  providersThrottleDuration: 2

  file:
    filename: traefik_dynamic_config.yml
    watch: true

  docker:
    watch: true
    exposedByDefault: false

api:
  insecure: true
  dashboard: true

log:
  level: INFO

certificatesResolvers:
  letsencrypt:
    acme:
      email: "{{ email }}"
      storage: acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

My traefik_dynamic_config.yml (here traefik.{{ domain_name }} works but traefik.local.{{ domain_name }} doesnt)

http:
  routers:
    traefik-dashboard:
      rule: "Host(`traefik.{{ domain_name }}`)"
      entryPoints:
        - https 
      middlewares:
        - "local-ipwhitelist"
        - "securityHeaders@file"
      service: api@internal

  middlewares:
    securityHeaders:
      headers:
        customResponseHeaders:
          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex"
          server: ""
          X-Forwarded-Proto: "https"
        sslProxyHeaders:
          X-Forwarded-Proto: "https"
        referrerPolicy: "strict-origin-when-cross-origin"
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        customRequestHeaders:
          X-Forwarded-Proto: "https"
        contentTypeNosniff: true
        browserXssFilter: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsSeconds: 63072000
        stsPreload: true

    local-ipwhitelist:
      IPAllowList:
        sourceRange:
          - '10.0.0.0/8'
          - '172.16.0.0/12'
          - '192.168.0.0/16'

Usually you can’t just use multi-level sub-domains with a provider. You need to ensure the parent level:

example.com
*.example.com
shop.example.com
*.shop.example.com

I usually recommend to place acme.json at a fixed path, so you can ensure to use a Docker bind mount or volume with that path to persist it. LetsEncrypt has limits which you will quickly reach during testing if you don’t correctly persist the file. Check simple Traefik example.

You seem to use some random settings. For example

is not usually used in response and it is automatically set by Traefik for https requests. Also X-Forwarded-Host is automatically set.

So I did try

example.com
*.example.com
*.shop.example.com

but it didnt have shop.example.com in it at the time. Will this be in domain or sans? Also even without the cert I think it should proxy right?

As for the acme.json it is persisted and handled in my compose. As for the settings I will need to look into it, I just copied someone's settings and called it a day havent gotten around to actually mucking around with it.

I would split it:

        domains:
          - main: "{{ domain_name }}"
            sans:
              - "*.{{ domain_name }}"
          - main: "shop.{{ domain_name }}"
            sans:
              - "*.shop.{{ domain_name }}"

And make sure shop. is explicitly registered as sub-domain with your DNS provider.

should this sub-domain be a A-Record or is a CNAME enough?