[solved] Traefik resolving cloudflare domain/sub-domains to an old machine IP - can't access dashboard either

[eskyyer]

Hello!

My setup:

• Hardware - Arch Linux VM (running on Truenas Core)
• Running traefik:latest on docker
• Cloudflare domain, exclusively for my home network services running as easy to parse sub-domains (plex, etc..)

So, I was running this docker-compose file with traefik on my old machine just fine for the last year. I was able to connect to local apps like plex with a simple plex.domain.com rather than a full network ip+port.

I am moving my system to a Truenas Core machine and decided to run my old docker compose setup on a linux VM, namely, Arch, which upon fully setup, is failing to resolve everything, I can't even access traefik admin page on any of the setup ports.

I've isolated traefik and portainer into a single docker compose, minimal setup, and I still can't access anything, yet, trying to ping any subdomain.domain.com always results in a 192.168.2.92 IP showing up

this was the former IP of my old machine on my home network, it's no longer running nor showing up assigned to any other machine in my network.

• I access traefik logs on portainer while its running, 0 errors.

Here's my docker-compose.yml

traefik:
    image: traefik:latest
    container_name: traefik
    command:
      - --api.insecure=true
      - --providers.docker
      - --api.dashboard=true
      - --log.level=DEBUG
    security_opt:
      - no-new-privileges:true
    networks:
      - media
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:     
      - CF_DNS_API_TOKEN=#####obfuscated for this post####
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/config/traefik.yml:/traefik.yml:ro
      - ./traefik/config/acme.json:/acme.json
      - ./traefik/config/config.yml:/config.yml:ro
    labels:
      - "traefik.enable=true"
      # middlewares
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      # routers
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.entrypoints=https"
      - "traefik.http.routers.traefik.tls=true"
      - "traefik.http.routers.traefik.tls.certresolver=cloudflare"
      - "traefik.http.routers.traefik.tls.domains[0].main=domain.com"
      - "traefik.http.routers.traefik.tls.domains[0].sans=*.domain.com"
      - "traefik.http.routers.traefik.service=api@internal"
      # itself
      - "traefik.http.routers.traefik.rule=Host(`dashboard.domain.com`)"
    restart: unless-stopped

this is my traefik.yml

api:
  dashboard: true
  debug: true
entryPoints:
  http:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https
  https:
    address: ":443"
serversTransport:
  insecureSkipVerify: true
providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
  file:
    filename: /config.yml
certificatesResolvers:
  cloudflare:
    acme:
      email: mycloudflare@login-email.com
      storage: acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

this is my config.yml (with just one router/service to test against, if I ping it in a terminal, it always resolves to 192.168.2.29)

http:
  routers:
    homeassistant:
      entryPoints:
        - "https"
      rule: "Host(`home.domain.com`)"
      tls:
        certResolver: cloudflare
      service: localdockerapp


  services:
    localdockerapp:
      loadBalancer:
        servers:
          - url: "http://192.168.2.50:8123"
        passHostHeader: true

this is my acme.json

{
  "cloudflare": {
    "Account": {
      "Email": " mycloudflare@login-email.com",
      "Registration": {
        "body": {
          "status": "valid",
          "contact": [
            "mailto: mycloudflare@login-email.com"
          ]
        },
        "uri": "https://acme-v02.api.letsencrypt.org/acme/acct/##number obfuscated##"
      },
      "PrivateKey": "##token obfuscated##",
      "KeyType": "4096"
    },
    "Certificates": [
      {
        "domain": {
          "main": "domain.com",
          "sans": [
            "*.domain.com"
          ]
        },
        "certificate": "##token obfuscated##",
        "key": "##token obfuscated##",
        "Store": "default"
      }
    ]
  }
}

Info:

• I can access any other docker container in this network just fine.
• I can ping any of my other VMs/Machines in the home network just fine.
• my Arch VM is exclusively running docker, the OS is pretty vanilla otherwise.
• I installed manjaro ARM on a raspberry PI to setup another docker instance, outside my VMs, and it also resulted in parsing that same IP, 192.168.2.29 when pinging ***.domain.com
• I am an enthusiastic/noob, I learn mostly from youtube and online tutorials until it works.

Questions:

• Does cloudflare/certificate retains any IP I need to flush?
• I can't access the traefik admin panel/dashboard to check my services, loadbalancers, etc... any pointers? 80, 443 results in a 404 error, and 8080 doesn't resolve at all.
• Is there any terminal command I can use to check what services are running and their respective IPs in my VM?

Thanks in advance!

[eskyyer]

Solved. I forgot a setting on nextDNS from my prev setup whom was broadcasting that IP across my network.