Facing issues upgrading my traefik setup to docker swarm

Traefik Traefik v2
1

I have two docker-compose.*.yml files, one for the testing stage and one for production. The testing stage file is executed with docker compose and the production with docker swarm.

The docker compose setup works fine. In case of the production docker swarm setup I am getting a timeout 504 http status code when accessing the rabbitmq management endpoint.

Since the logs of both containers, traefik as well as rabbitmq do not display any error I do not know how to debug this.

Here are both files:

docker-compose.testing-stage.yml
(working example, executed with docker compose)

version: '3.7'

services:
  traefik:
    image: traefik:v2.2
    hostname: traefik
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /root/acme.json:/root/acme.json
      - /root/credentials.txt:/root/credentials.txt
    ports:
      - 80:80
      - 443:443
    command:
      - --api=true
      - --log.level=WARN
      - --providers.docker=true
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --providers.docker.exposedByDefault=false
      - --certificatesresolvers.secure.acme.httpchallenge=true
      - --certificatesresolvers.secure.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.secure.acme.email=${MAIL_ADDRESS}
      - --certificatesresolvers.secure.acme.storage=/root/acme.json
    labels:
      - traefik.enable=true
      # dashboard
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.routers.traefik.rule=Host(`monitor.example.org`)
      - traefik.http.routers.traefik.tls.certresolver=secure
      - traefik.http.routers.traefik.middlewares=auth
      - traefik.http.services.traefik.loadbalancer.server.port=8080
      - traefik.http.middlewares.auth.basicauth.usersfile=/root/credentials.txt
      # https redirect
      - traefik.http.routers.detour.rule=hostregexp(`{host:[a-z-.]+}`)
      - traefik.http.routers.detour.entrypoints=web
      - traefik.http.routers.detour.middlewares=redirect-to-https
      - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
      - traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https
      - traefik.http.services.dummy-svc.loadbalancer.server.port=9999
  rabbitmq:
    image: registry.exampe.com/root/blicc/rabbitmq:test
    hostname: rabbitmq
    environment:
      - RABBITMQ_ERLANG_COOKIE=${RABBITMQ_PASSWORD}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASSWORD}
      - RABBITMQ_DEFAULT_USER=admin
    ports:
      - 15672:15672
    labels:
      - traefik.enable=true
      - traefik.http.routers.rabbitmq.rule=Host(`messaging.example.org`)
      - traefik.http.routers.rabbitmq.tls.certresolver=secure
      - traefik.http.services.rabbitmq.loadbalancer.server.port=15672

docker-compose.prod.yml
(example which gives a timeout on messaging.prod-example.org, executed with docker swarm)

version: '3.7'

services:
  traefik:
    image: traefik:v2.2
    hostname: traefik
    ports:
      - 80:80
      - 443:443
    command:
      # entry points
      - --api=true
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      # tls certificates
      - --certificatesresolvers.secure.acme.httpchallenge=true
      - --certificatesresolvers.secure.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.secure.acme.email=${MAIL_ADDRESS}
      - --certificatesresolvers.secure.acme.storage=/root/acme.json
      # metrics
      - --metrics=true
      - --metrics.prometheus=true
      # docker
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false
      - --providers.docker.swarmMode=true
      - --providers.docker.network=traefik-public
      - --providers.docker.endpoint=unix:///var/run/docker.sock
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /root/acme.json:/root/acme.json
      - /root/credentials.txt:/root/credentials.txt
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        order: start-first
        failure_action: rollback
        delay: 10s
      rollback_config:
        parallelism: 0
        order: stop-first
      restart_policy:
        condition: any
        delay: 5s
        max_attempts: 3
        window: 120s
      placement:
        constraints:
          - node.role == manager
      labels:
        - traefik.enable=true
        # dashboard
        - traefik.http.routers.traefik.service=api@internal
        - traefik.http.routers.traefik.rule=Host(`monitor.prod-example.org`)
        - traefik.http.routers.traefik.tls.certresolver=secure
        - traefik.http.routers.traefik.middlewares=auth
        - traefik.http.middlewares.auth.basicauth.usersfile=/root/credentials.txt
        - traefik.http.services.traefik.loadbalancer.server.port=8080
        # https redirect
        - traefik.http.routers.detour.rule=hostregexp(`{host:[a-z-.]+}`)
        - traefik.http.routers.detour.entrypoints=web
        - traefik.http.routers.detour.middlewares=redirect-to-https
        - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
        - traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https
        - traefik.http.services.dummy-svc.loadbalancer.server.port=9999
  rabbitmq:
    image: registry.exampe.com/root/blicc/rabbitmq:latest
    hostname: rabbitmq
    environment:
      - RABBITMQ_ERLANG_COOKIE=${RABBITMQ_PASSWORD}
      - RABBITMQ_DEFAULT_PASS=${RABBITMQ_PASSWORD}
      - RABBITMQ_DEFAULT_USER=admin
    ports:
      - 15672:15672
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        order: start-first
        failure_action: rollback
        delay: 10s
      rollback_config:
        parallelism: 0
        order: stop-first
      restart_policy:
        condition: any
        delay: 5s
        max_attempts: 3
        window: 120s
      placement:
        constraints:
          - node.role == manager
      labels:
        - traefik.enable=true
        - traefik.http.routers.rabbitmq.rule=Host(`messaging.prod-example.org`)
        - traefik.http.routers.rabbitmq.tls.certresolver=secure
        - traefik.http.services.rabbitmq.loadbalancer.server.port=15672

Both server run the ubuntu 18.04 with the same firewall and the same ports exposed. I am guessing that I do some mistakes on the docker swarm setup for traefik, but I can not figure out what. The only thing I basically changed was putting the labels under deploy.

The rabbitmq container has the ui exposed on port 15672 which I am mapping with the load balancer to port 443 on messaging.prod-example.org. Nevertheless this endpoint gives me an timeout.

Does anyone sees the misconfiguration I am doing here?