Continuous 404 instead of dashboard and error 521 with Clouadflare

I have tried to follow several walkthroughs and use configs that have been published that should be working, then adapted them to what I needed. I have all three configs (docker-compose.yml, traefik.yml, config.yml). I am currently not interested in having Traefik set records and actually alter my DNS, only have it verify and I don't want it to spin up entries if a new docker is added. I want more manual control. The files can be found on pastebin as reddit alters the spacing (so did vscode set to use yaml, so the document was created as a standard txt. Each section is separated by about 5-7 new lines and a comment naming the file. https://pastebin.com/5Uyn3qSD

Any suggestions would help. I feel if I can get both the dashboard and say my vaultwarden instance running properly that I can figure out the rest of my domains. The final goal is to have tailscale setup and route most traffic from my cloud VPS through tailscale using local IP addresses for the services. That way, I can eliminate my home server IP on DNS records.

As far as errors with the current configuration, they are as follows from the docker logs

time="2022-11-13T07:17:03Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
time="2022-11-13T08:21:25Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:21:25Z" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:21:25Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
time="2022-11-13T08:30:40Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:30:40Z" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:30:41Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
time="2022-11-13T08:36:35Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:36:35Z" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2022-11-13T08:36:36Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"

No logs ever get created under /var/log/traefik or on the host side appdata/traefik/logs

the docker logs do not appear to show anything about websecure, dashboard, or loading from config.yml. Not sure why. Nothing changes in the acme.json as well.

Bumping in hopes of getting some help.

You bump after 20 hours? :upside_down_face:

Remove the log filepath, then you get the log output on the container stdout.

Yeah, I noticed after 24 hours, stuff starts getting missed, and people don't scroll back far enough. But I tried to do it before 24 hours so that maybe others may see it before that wouldn't have seen it. I will remove that and try again and see what happens.

Have you looked at the v2 group? :sweat_smile: Posts are like 6 days old and still on the first page. Not so much traffic here. And I promise you bumbing will not help you here.

Disabled the logging lines and it broke and stopped running. Got it back up but it is still doing the same thing. 404 errors on my server no matter what including the dashboard should be internal.

You removed

filePath: "/var/log/traefik/traefik.log"

and it broke? Meaning the container crashed? No debug log?

Yes, when I removed the filepath it wouldn't start with an error message. I then removed all logging since by default it is supposed to log to console. I am getting nothing though.

That's like my mom when her computer does not work. There was an error message, she clicked it away :joy:

What was the error? Can you just enable Traefik debug log on stdout?

I actually just added it back in to check, I am getting some logging with debug now to the docker logs. I am not seeing it thorough enough as to why I am getting a 404 though on every page.

Yeah it is bad lol. Can't look at anything without an error message. I got them working. The bad thing is it looks like I need to look into documentation on the certificateResolver. It isn't generating a a cert and just trying to give the default one out.

There are similar unresolved posts: 1, 2. Someone seems to have resolved it by pinning an old version. You can try, but I wouldn't really recommend it. Is docker compose always pulling latest? Maybe try pulling traefik:latest manually.

Yes, my docker-compose is always pulling the latest. I am getting just 404 and it looks like that closed only happens on the first round, then it sees them open. I can reach traefik but I get 404's. If I go to my bitwarden address, I have a cloudflare cert and see the 404 error page. If I go to my traefik dashboard address, I get a Traefik Default cert. It doesn't appear to be trying to get me a certificate for any domain.

inside traefik.yml - static config

# Configure your CertificateResolver here...
# ---
certificatesResolvers:
#  staging:
#    acme:
#      email: redacted
#      storage: /etc/traefik/certs/acme.json
#      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
#      httpChallenge:
#        entryPoint: web

  production:
    acme:
      email: "redacted"
      storage: "/etc/traefik/certs/acme.json"
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      dnsChallenge:
        provider: cloudflare
          - CF_DNS_API_TOKEN=${CF_TOKEN}

config.yml in /etc/traefik/config/

  services:
    #vpn server
    vpn:
      loadBalancer:
        servers:
          - url: https://10.133.122.220
          - url: https://home.tld
    
    # Bitwarden instance
    vaultwarden-ser:
      loadBalancer:
        servers:
          - url: http://vaultwarden #can be pinged from inside the container also tried https. So I know it is on the right docker network.
  
  routers:
    #Vaultwarden Route
    vaultwarden-route:
      entrypoints:
        - "websecure"
        - "web"
      rule: "Host(`bitwarden.tld`)"
#      middlewares:
#        - default
      tls:
        certResolver: production
        domains:
          - main: "bitwarden.tld"
            sans:
              - "*.tld"
      service: "vaultwarden-ser"

    #Dashboard
    dashboard:
      entrypoints: 
        - "web"
        - "websecure"
      rule: Host(`traefik.tld`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
      service: api@internal
#      middlewares:
#        - traefikAuth

My errors do not show any attempt at creating a cert, and the acme.json is empty.

Forget the certificateResolver. It appears that my instance refuses to use load the dynamic. I see in the log that it says it is watching the directory, but the logs never say that /etc/traefik/config/config.yml ever gets loaded.

Make sure the certResolver is created and that the right one is assigned to your routers.

Yeah in the logs it's made, it does a test. But Traefik isn't reading and loading my dynamic config.

Go into your container and check that the file exists, is readable and has the content you expect.

docker exec -it traefik sh
cat /etc/traefik/config.yml

Yup, I can reach the file inside the container.

Check again that your paths are all correct

Here is my volume mapping out of the docker-compose, it is kinda redundant as I map all of my ~/appdata/traefik to /etc/traefik so it should see everything in the directory.

    volumes:
      - /home/dockeruser/appdata/traefik:/etc/traefik
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /home/dockeruser/appdata/traefik/logs:/var/log/traefik
      - /home/dockeruser/appdata/traefik/certs:/etc/traefik/certs
      - /home/dockeruser/appdata/traefik/config:/etc/traefik/config

Here is output from the console showing that snippets that all the paths should be correct unless I am missing something.

/ # cd /etc/traefik
/etc/traefik # ls
certs        config       logs         traefik.yml
/etc/traefik # ls config
config.yml
/etc/traefik # tail -n 5 traefik.yml 
    # watch for dynamic configuration changes
    directory: /etc/traefik/config
    watch: true


/etc/traefik # head /etc/traefik/config/config.yml 
# Traefik dynamic configuration file
# See https://doc.traefik.io/traefik/getting-started/configuration-overview/#the-dynamic-configuration

https:
  middlewares:
#    # A basic authentification middleware, to protect the Traefik dashboard to anyone except myself
#    # Use with traefik.http.routers.myRouter.middlewares: "traefikAuth@file"
    #traefikAuth:
#      basicAuth:
#        users: