Traefik gateway timeout for every service

Hey! I've recently set up my first docker swarm cluster, and i want to deploy a traefik stack aswell for loadbalancing and as a reverse proxy.

i've set up traefik and the overlay network for it largely like it's described in Traefik Proxy with HTTPS - Docker Swarm Rocks , and traefik mostly works (letsencrypt and service discovery via labels) However, the proxied services always show a gateway timeout, and that leads me to believe that somehow the different stacks can't talk to each other.

Maybe someone can give me a hint as to what i'm missing here? I didnt find anything about this specific issue, so i'm posting here.
Thank you all in advance <3

Overlay network: docker network create traefik-public --driver overlay

traefik docker-compose:

version: "3.6"

services:
  traefik:
    image: traefik:v2.5
    environment:
      - "OVH_ENDPOINT=ovh-eu"
      - "OVH_APPLICATION_KEY=xxx"
      - "OVH_APPLICATION_SECRET=xxxx"
      - "OVH_CONSUMER_KEY=xxxx"
    ports:
      - 80:80
      - 443:443
    deploy:
      replicas: 2
      placement:
        constraints:
          - node.role == manager
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.middlewares.admin-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
        - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
        - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
        - traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?Variable not set}`)
        - traefik.http.routers.traefik-public-http.entrypoints=http
        - traefik.http.routers.traefik-public-http.middlewares=https-redirect
        - traefik.http.routers.traefik-public-https.rule=Host(`${DOMAIN?Variable not set}`)
        - traefik.http.routers.traefik-public-https.entrypoints=https
        - traefik.http.routers.traefik-public-https.tls=true
        - traefik.http.routers.traefik-public-https.service=api@internal
        - traefik.http.routers.traefik-public-https.tls.certresolver=le
        - traefik.http.routers.traefik-public-https.middlewares=admin-auth
        - traefik.http.services.traefik-public.loadbalancer.server.port=8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - "/var/docker-storage/traefik/certificates:/certificates"
    command:
      - --providers.docker
      - --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
      - --providers.docker.exposedbydefault=false
      - --providers.docker.swarmmode
      - --entrypoints.http.address=:80
      - --entrypoints.https.address=:443
      - --certificatesresolvers.le.acme.email=admin@my.domain
      - --certificatesresolvers.le.acme.storage=/certificates/acme.json
      - --certificatesresolvers.le.acme.dnschallenge=true
      - --certificatesresolvers.le.acme.dnschallenge.provider=ovh
      - --certificatesresolvers.le.acme.dnschallenge.resolvers=1.1.1.1:53,8.8.8.8:53
      - --accesslog
      - --log
      - --api
    networks:
      - traefik-public


networks:
  traefik-public:
    external: true
    name: traefik-public

whoami docker-compose:

version: '3.6'
services:
  whoami:
    image: traefik/whoami:latest
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.whoami-http.rule=Host(`whoami.example.com`)
        - traefik.http.routers.whoami-http.entrypoints=http
        - traefik.http.routers.whoami-http.middlewares=https-redirect
        - traefik.http.routers.whoami-https.rule=Host(`whoami.example.com`)
        - traefik.http.routers.whoami-https.entrypoints=https
        - traefik.http.routers.whoami-https.tls=true
        - traefik.http.routers.whoami-https.tls.certresolver=le
        - traefik.http.services.whoami.loadbalancer.server.port=80
    networks:
      - traefik-public

networks:
  traefik-public:
    external: true
    name: traefik-public

Hi, so for anyone having the same issue, this is what I've learned in the meantime and how i fixed it.

My docker swarm cluster is running on VMware ESXi. Docker swarm uses 4789/udp for the overlay data port. ESXi hosts seem to block outgoing 4789/udp, because that's their VTEP VXLAN port for NSX-T.

To work around that, I recreated my swarm with docker swarm init --data-path-port=7789, and now everything works as intended.

Stackoverflow post for more details: networking - Docker-swarm overlay network is not working for containers in different hosts - Stack Overflow

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.