404 even when route exists

I am running into a dead end. I have a number of services defined and one of them is not working and i cannot tell why.

In the config, I am able to access everything except for either qBittorrent or sabnzbd and a lot seems to depend on which one was started last

Note: I am running traefik v2.2 on docker on ubuntu 18.4

Here is the output of the routes and services from the log file

{
  "http": {
    "routers": {
      "pihole": {
        "entryPoints": [
          "websecure"
        ],
        "service": "pihole",
        "rule": "Host(`pihole.home.local`)",
        "tls": {}
      },
      "portainer": {
        "entryPoints": [
          "websecure"
        ],
        "service": "portainer",
        "rule": "Host(`portainer.home.local`)",
        "tls": {}
      },
      "qBittorrent": {
        "entryPoints": [
          "websecure"
        ],
        "service": "qBittorrent",
        "rule": "Host(`qbittorrent.home.local`)",
        "tls": {}
      },
      "sabnzbd": {
        "entryPoints": [
          "websecure"
        ],
        "service": "sabnzbd",
        "rule": "Host(`sabnzbd.home.local`)",
        "tls": {}
      },
      "traefik-dashboard": {
        "entryPoints": [
          "traefik"
        ],
        "service": "api@internal",
        "rule": "Host(`traefik.home.local`) \\u0026\\u0026 (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      },
      "traefik-whoami": {
        "entryPoints": [
          "websecure"
        ],
        "service": "traefik-whoami",
        "rule": "Host(`traefik-whoami.home.local`)",
        "tls": {}
      }
    },
    "services": {
      "pihole": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.24.0.2:80"
            }
          ],
          "passHostHeader": true
        }
      },
      "portainer": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.22.0.2:9000"
            }
          ],
          "passHostHeader": true
        }
      },
      "qBittorrent": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.20.0.4:8081"
            }
          ],
          "passHostHeader": true
        }
      },
      "sabnzbd": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.20.0.3:8080"
            }
          ],
          "passHostHeader": true
        }
      },
      "traefik-traefik": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.21.0.2:80"
            }
          ],
          "passHostHeader": true
        }
      },
      "traefik-whoami": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://172.18.0.2:80"
            }
          ],
          "passHostHeader": true
        }
      }
    }
  },
  "tcp": {},
  "udp": {}
}

and here is my compose file

version: "3.7"

services:

  traefik:
    container_name: traefik
    image: traefik:v2.2
    env_file: 
      - .env
      - .env.traefik
    volumes:
      - ./data/certs:/certs
      - ./data/logs:/logs
      - ./data/letsencrypt:/letsencrypt
      - ./init/config:/etc/traefik/dynamic
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - public 
      - data 
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    command: 
      - --log.level=DEBUG
      - --log.filePath=/logs/traefik.log
      - --log.format=json
      - --accesslog=true
      - --accesslog.filepath=/logs/access.log
      - --api.dashboard=true
      - --api.insecure=true
      - --providers.docker=true
      - --providers.docker.endpoint=unix:///var/run/docker.sock
      - --providers.docker.exposedbydefault=false
      - --providers.docker.useBindPortIP=false
      - --providers.file.directory=/etc/traefik/dynamic
      - --providers.file.watch=true
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --certificatesresolvers.secureresolver.acme.httpchallenge=true
      - --certificatesresolvers.secureresolver.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.secureresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory
      - --certificatesresolvers.secureresolver.acme.email=certs@domain.com
      - --certificatesresolvers.secureresolver.acme.storage=/letsencrypt/acme.json
      - --serverstransport.insecureskipverify=true
    security_opt:
      - no-new-privileges:true
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-dashboard.entryPoints=traefik"
      - "traefik.http.routers.traefik-dashboard.rule=Host(`${CONTAINER_APP_NAME_TRAEFIK}.${CONTAINER_DOMAIN}`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      - "traefik.http.routers.traefik-dashboard.service=api@internal"
    restart: always

Any chance anyone can see what I am doing wrong?

May be related: V2: Intermittent 404 errors across our docker containers

That could be it -- I rolled back to 2.2.1 and everything started working.

If you feel you can contribute to: Consolidated 404 issues thread in versions since 2.2.1 that would be awesome! Output of the routes and services from the log file is difficult to translate to initial configuration, and it looks like your compose file includes only traefik, and not the rest of the containers you are running, also it appears that you have something in ./init/config:/etc/traefik/dynamic that you are not showing that would be required for reproducing this issue. If you feel you have time to help out here, that would be most appreciated.

I'll see what I can add. As for the dynamic folder, there isn't much there as I just started playing with the files

Here is the one file in there

api:
  insecure: true
  dashboard: true

As for the rest of the containers -- there is a number of them :slight_smile: You can see the configs here