Traefik unable to pick up routers/services and 404s when accessing API container (openSUSE Tumbleweed)

I’ve got a docker compose setup including traefik which is working fine on Ubuntu (in WSL2) and macOS, but I recently switched to openSUSE Tumbleweed and no matter what I try, I get traefik’s 404 not found response when trying to access my API. I’ve confirmed that I can get the correct response if I directly hit the API with docker exec -it backend sh -c “curl localhost/api/v2/utils/health”. I’ve also tried out the basic whoami example from the traefik site and I still get a 404. Below are my docker-compose.yml and docker-compose.override.yml files. Wondering if there’s anyone that can give me a hand. I’ve tried a ton of things I’ve seen online, including stopping the firewalld service to see if it’s related to that, adding the networks in docker-compose.yml to the backend (the API) container, among other things.

I’m pretty bad with networking stuff, but there were a couple things I thought were odd when I started digging. Just default Tumbleweed stuff, seemingly. There was no static hostname when looking at hostnamectl. Only a transient of localhost. /etc/hostname was set to localhost, though. But I set the hostname to localhost so that’s static now. There was also a fallback to 127.0.0.1 in /etc/hosts to localhost.localdomain which I removed. Not sure if it makes sense to do this or not.

I’ve also confirmed that the server in the backend container is running on 0.0.0.0:80 (within the container). I’m also in the docker group on my machine, and am running rootless.

At this point I’m sort of thinking it’s just some Docker/OS config that I don’t have set up correctly, but not sure.

docker-compose.yml

version: "3.8"

services:
  proxy:
    image: traefik:v2.10
    networks:
      - ${TRAEFIK_PUBLIC_NETWORK?TRAEFIK_PUBLIC_NETWORK variable not set}
      - default
    env_file:
      - .env
    environment:
      - STACK_NAME=${STACK_NAME?STACK_NAME variable not set}
      - TRAEFIK_TAG=${TRAEFIK_TAG?TRAEFIK_TAG variable not set}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command:
      # Enable Docker in Traefik, so that it reads labels from Docker services
      - --providers.docker
      # Add a constraint to only use services with the label for this stack
      # from the env var TRAEFIK_TAG
      - --providers.docker.constraints=Label(`traefik.constraint-label-stack`, `${TRAEFIK_TAG?TRAEFIK_TAG variable not set}`)
      # Do not expose all Docker services, only the ones explicitly exposed
      - --providers.docker.exposedbydefault=false
      # Enable Docker Swarm mode
      - --providers.docker.swarmmode
      # Enable the access log, with HTTP requests
      - --accesslog
      # Enable the Traefik log, for configurations and errors
      - --log
      # Enable the Dashboard and API
      - --api
  # removed deploy.labels here since it's not useful

  backend:
    image: "XXXXX/${DOCKER_IMAGE_BACKEND?DOCKER_IMAGE_BACKEND variable not set}:${TAG-latest}"
    env_file:
      - .env
    environment:
      # ...
      # bunch of envars
      # ...
    build:
      context: ./backend
      args:
        INSTALL_DEV: ${INSTALL_DEV-false}
    # removed deploy.labels here since it's not useful

volumes:
  app-db-data:

networks:
  traefik-public:
    # Allow setting it to false for testing
    external: ${TRAEFIK_PUBLIC_NETWORK_IS_EXTERNAL-true}

docker-compse.override.yml

version: "3.8"

services:
  proxy:
    ports:
      - "80:80"
      - "8090:8080"
    command:
      # Enable Docker in Traefik, so that it reads labels from Docker services
      - --providers.docker
      # Add a constraint to only use services with the label for this stack
      # from the env var TRAEFIK_TAG
      - --providers.docker.constraints=Label(`traefik.constraint-label-stack`, `${TRAEFIK_TAG?TRAEFIK_TAG not set}`)
      # Do not expose all Docker services, only the ones explicitly exposed
      - --providers.docker.exposedbydefault=false
      # Disable Docker Swarm mode for local development
      # - --providers.docker.swarmmode
      # Enable the access log, with HTTP requests
      - --accesslog
      # Enable the Traefik log, for configurations and errors
      - --log
      - --log.level=DEBUG
      # Enable the Dashboard and API
      - --api
      # Enable the Dashboard and API in insecure mode for local development
      - --api.insecure=true
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?DOMAIN not set}`)
      - traefik.http.services.traefik-public.loadbalancer.server.port=80

  backend:
    ports:
      - "8888:8888"
    volumes:
      - ./backend/app:/app
    environment:
      - SERVER_HOST=http://${DOMAIN?DOMAIN not set}
    build:
      context: ./backend
      args:
        INSTALL_DEV: ${INSTALL_DEV-true}
    command: /start-reload.sh
    labels:
      - traefik.enable=true
      - traefik.constraint-label-stack=${TRAEFIK_TAG?TRAEFIK_TAG not set}
      - traefik.http.routers.backend-http.rule=PathPrefix(`/api`) || PathPrefix(`/docs`) || PathPrefix(`/redoc`)
      - traefik.http.services.backend.loadbalancer.server.port=80

networks:
  traefik-public:
    # For local dev, don't expect an external Traefik network
    external: false

Don’t understand your override complication.

You should explicitly connect Traefik and the target service to the same Docker network.

Compare with simple Traefik example.

You enable Docker Swarm on Traefik, but seemingly don’t run Swarm as you build an image inside compose. If you don’t use it, don’t enable it.

Thanks for the quick reply!

My docker-compose.yml sets Swarm mode and my override disables it, as I don't want to use Swarm locally - it's used when I deploy, but my issue is only happening locally.

I tried explicitly including the backend service in the same networks as the proxy service, but that didn't seem to affect anything (the 404s remained). I confirmed that they're in the same default network through an inspect, though, when not explicitly setting it.

I'll give that example a closer look, but I wasn't able to get the even more basic one from the Traefik site to work - it still 404s.