Traefik gateway timeout

Traefik Traefik v2
1

Hi,

When using traefik with docker, I get some erratic "gateway timeout" when I'm trying to access it.

In the logs I see this:

time="2023-03-12T12:14:12Z" level=debug msg="Trying to challenge certificate for domain [medusa.xxx.yyy] found in HostSNI rule" rule="Host(`medusa.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=medusa@docker

time="2023-03-12T12:14:12Z" level=debug msg="Looking for provided certificate(s) to validate [\"medusa.xxx.yyy\"]..." providerName=myresolver.acme routerName=medusa@docker rule="Host(`medusa.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory"

time="2023-03-12T12:14:12Z" level=debug msg="Looking for provided certificate(s) to validate [\"home.xxx.yyy\"]..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme rule="Host(`home.xxx.yyy`)" routerName=heimdall@docker

time="2023-03-12T12:14:12Z" level=debug msg="No ACME certificate generation required for domains [\"ddns.xxx.yyy\"]." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=ddns-updater@docker rule="Host(`ddns.xxx.yyy`)"

time="2023-03-12T12:14:12Z" level=debug msg="No ACME certificate generation required for domains [\"medusa.xxx.yyy\"]." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=medusa@docker rule="Host(`medusa.xxx.yyy`)"

time="2023-03-12T12:14:12Z" level=debug msg="No ACME certificate generation required for domains [\"home.xxx.yyy\"]." routerName=heimdall@docker ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme rule="Host(`home.xxx.yyy`)"

time="2023-03-12T12:14:13Z" level=debug msg="Provider event received {Status:start ID:d9478a3d82cd664974007d048043d123ee76da5261d28be3541a8e6e5f09910a From:lscr.io/linuxserver/emby:latest@sha256:735d90f69d6f35660933521316eb427521df2a1e8b4dd44adc4951b986c25996 Type:container Action:start Actor:{ID:d9478a3d82cd664974007d048043d123ee76da5261d28be3541a8e6e5f09910a Attributes:map[build_version:Linuxserver.io version:- 4.7.11.0-ls147 Build-date:- 2023-03-12T10:50:54+01:00 com.docker.stack.namespace:home-stack com.docker.swarm.node.id:mjcnxi238xhu9ahrvjh79v5sb com.docker.swarm.service.id:777n19d4h7r5czr9fipsaokbj com.docker.swarm.service.name:home-stack_emby com.docker.swarm.task: com.docker.swarm.task.id:2a3xhkhdjnetbbm2khtpo88kx com.docker.swarm.task.name:home-stack_emby.1.2a3xhkhdjnetbbm2khtpo88kx image:lscr.io/linuxserver/emby:latest@sha256:735d90f69d6f35660933521316eb427521df2a1e8b4dd44adc4951b986c25996 maintainer:thelamer name:home-stack_emby.1.2a3xhkhdjnetbbm2khtpo88kx org.opencontainers.image.authors:linuxserver.io org.opencontainers.image.created:2023-03-12T10:50:54+01:00 org.opencontainers.image.description:[Emby](https://emby.media/) organizes video, music, live TV, and photos from personal media libraries and streams them to smart TVs, streaming boxes and mobile devices. This container is packaged as a standalone emby Media Server. org.opencontainers.image.documentation:https://docs.linuxserver.io/images/docker-emby org.opencontainers.image.licenses:GPL-3.0-only org.opencontainers.image.ref.name:d55db1cefa3bec6cd231769cfedd3863e766ff8b org.opencontainers.image.revision:d55db1cefa3bec6cd231769cfedd3863e766ff8b org.opencontainers.image.source:https://github.com/linuxserver/docker-emby org.opencontainers.image.title:Emby org.opencontainers.image.url:https://github.com/linuxserver/docker-emby/packages org.opencontainers.image.vendor:linuxserver.io org.opencontainers.image.version:4.7.11.0-ls147 traefik.enable:true traefik.http.routers.emby.entrypoints:websecure traefik.http.routers.emby.rule:Host(`emby.xxx.yyy`) traefik.http.routers.emby.tls.certresolver:myresolver traefik.http.services.emby.loadbalancer.server.port:8096]} Scope:local Time:1678623253 TimeNano:1678623253517083891}" providerName=docker

time="2023-03-12T12:14:13Z" level=debug msg="Filtering disabled container" providerName=docker container=portainer-agent-mjcnxi238xhu9ahrvjh79v5sb-r8xfqpd0s2abfkchu5ixx19xa-73a34550a967da2cc0752ce16fc059a898d315ce26e0d096cf7febfe556aa22f

time="2023-03-12T12:14:13Z" level=debug msg="Filtering disabled container" providerName=docker container=portainer-portainer-1-npovb4azpsh4jaenc4k030o4l-db5d3f7ce7bba2eba6936263ceb2dfb3103165798116d1289dc8eea2c3fbf2ba

time="2023-03-12T12:14:14Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"ddns-updater\":{\"entryPoints\":[\"websecure\"],\"service\":\"ddns-updater\",\"rule\":\"Host(`ddns.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}},\"emby\":{\"entryPoints\":[\"websecure\"],\"service\":\"emby\",\"rule\":\"Host(`emby.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}},\"heimdall\":{\"entryPoints\":[\"websecure\"],\"service\":\"heimdall\",\"rule\":\"Host(`home.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}},\"http-catchall\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"force-secure\"],\"service\":\"traefik\",\"rule\":\"HostRegexp(`{any:.+}`)\"},\"medusa\":{\"entryPoints\":[\"websecure\"],\"service\":\"medusa\",\"rule\":\"Host(`medusa.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}},\"traefik\":{\"entryPoints\":[\"websecure\"],\"service\":\"traefik\",\"rule\":\"Host(`traefik.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}},\"transmission\":{\"entryPoints\":[\"websecure\"],\"service\":\"transmission\",\"rule\":\"Host(`transmission.xxx.yyy`)\",\"tls\":{\"certResolver\":\"myresolver\"}}},\"services\":{\"ddns-updater\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.3.39:8007\"},{\"url\":\"http://10.0.3.3:8007\"}],\"passHostHeader\":true}},\"emby\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.0.188:8096\"}],\"passHostHeader\":true}},\"heimdall\":{\"loadBalancer\":{\"servers\":[{\"url\":\"https://10.0.0.186:443\"}],\"passHostHeader\":true}},\"medusa\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.0.165:8081\"}],\"passHostHeader\":true}},\"traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.3.32:8080\"}],\"passHostHeader\":true}},\"transmission\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.3.31:9091\"}],\"passHostHeader\":true}}},\"middlewares\":{\"force-secure\":{\"redirectScheme\":{\"scheme\":\"https\",\"permanent\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) traefik.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) ddns.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) emby.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) home.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) medusa.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) heimdall.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="Adding certificate for domain(s) transmission.xxx.yyy"

time="2023-03-12T12:14:14Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware api@internal" entryPointName=traefik routerName=api@internal middlewareName=tracing middlewareType=TracingForwarder

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" middlewareName=dashboard_stripprefix@internal middlewareType=StripPrefix entryPointName=traefik routerName=dashboard@internal

time="2023-03-12T12:14:14Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_stripprefix@internal entryPointName=traefik routerName=dashboard@internal

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik routerName=dashboard@internal

time="2023-03-12T12:14:14Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex

time="2023-03-12T12:14:14Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_redirect@internal routerName=dashboard@internal entryPointName=traefik

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" middlewareName=pipelining middlewareType=Pipelining entryPointName=web routerName=http-catchall@docker serviceName=traefik

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=http-catchall@docker serviceName=traefik

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.3.32:8080" entryPointName=web routerName=http-catchall@docker serviceName=traefik serverName=0

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.3.32:8080 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware traefik" routerName=http-catchall@docker entryPointName=web middlewareName=tracing middlewareType=TracingForwarder

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" routerName=http-catchall@docker entryPointName=web middlewareName=force-secure@docker middlewareType=RedirectScheme

time="2023-03-12T12:14:14Z" level=debug msg="Setting up redirection to https " middlewareType=RedirectScheme routerName=http-catchall@docker entryPointName=web middlewareName=force-secure@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" serviceName=heimdall middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure routerName=heimdall@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=heimdall@docker serviceName=heimdall

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 https://10.0.0.186:443" serviceName=heimdall serverName=0 entryPointName=websecure routerName=heimdall@docker

time="2023-03-12T12:14:14Z" level=debug msg="child https://10.0.0.186:443 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware heimdall" middlewareType=TracingForwarder routerName=heimdall@docker entryPointName=websecure middlewareName=tracing

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" middlewareType=Pipelining entryPointName=websecure routerName=emby@docker serviceName=emby middlewareName=pipelining

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" serviceName=emby entryPointName=websecure routerName=emby@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.0.188:8096" entryPointName=websecure routerName=emby@docker serviceName=emby serverName=0

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.0.188:8096 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware emby" middlewareName=tracing middlewareType=TracingForwarder entryPointName=websecure routerName=emby@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" routerName=transmission@docker serviceName=transmission middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" routerName=transmission@docker serviceName=transmission entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.3.31:9091" serverName=0 routerName=transmission@docker serviceName=transmission entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.3.31:9091 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware transmission" middlewareType=TracingForwarder routerName=transmission@docker entryPointName=websecure middlewareName=tracing

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" routerName=ddns-updater@docker serviceName=ddns-updater middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=ddns-updater@docker serviceName=ddns-updater

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.3.3:8007" entryPointName=websecure routerName=ddns-updater@docker serviceName=ddns-updater serverName=0

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.3.3:8007 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 1 http://10.0.3.39:8007" entryPointName=websecure routerName=ddns-updater@docker serviceName=ddns-updater serverName=1

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.3.39:8007 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Still UP, no need to propagate"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware ddns-updater" entryPointName=websecure routerName=ddns-updater@docker middlewareName=tracing middlewareType=TracingForwarder

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" middlewareType=Pipelining entryPointName=websecure routerName=medusa@docker serviceName=medusa middlewareName=pipelining

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" serviceName=medusa entryPointName=websecure routerName=medusa@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.0.165:8081" serverName=0 entryPointName=websecure routerName=medusa@docker serviceName=medusa

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.0.165:8081 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware medusa" routerName=medusa@docker entryPointName=websecure middlewareName=tracing middlewareType=TracingForwarder

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" serviceName=traefik middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure routerName=traefik@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating load-balancer" serviceName=traefik entryPointName=websecure routerName=traefik@docker

time="2023-03-12T12:14:14Z" level=debug msg="Creating server 0 http://10.0.3.32:8080" serverName=0 entryPointName=websecure routerName=traefik@docker serviceName=traefik

time="2023-03-12T12:14:14Z" level=debug msg="child http://10.0.3.32:8080 now UP"

time="2023-03-12T12:14:14Z" level=debug msg="Propagating new UP status"

time="2023-03-12T12:14:14Z" level=debug msg="Added outgoing tracing middleware traefik" entryPointName=websecure routerName=traefik@docker middlewareName=tracing middlewareType=TracingForwarder

time="2023-03-12T12:14:14Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=traefik-internal-recovery middlewareType=Recovery

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for medusa.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for traefik.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for transmission.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for ddns.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for emby.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Adding route for home.xxx.yyy with TLS options default" entryPointName=websecure

time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [traefik.xxx.yyy] found in HostSNI rule" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=traefik@docker rule="Host(`traefik.xxx.yyy`)"

time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [home.xxx.yyy] found in HostSNI rule" routerName=heimdall@docker rule="Host(`home.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme

time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [medusa.xxx.yyy] found in HostSNI rule" routerName=medusa@docker rule="Host(`medusa.xxx.yyy`)" providerName=myresolver.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"

time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [emby.xxx.yyy] found in HostSNI rule" providerName=myresolver.acme routerName=emby@docker rule="Host(`emby.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory"

time="2023-03-12T12:14:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.xxx.yyy\"]..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=traefik@docker rule="Host(`traefik.xxx.yyy`)"

time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [transmission.xxx.yyy] found in HostSNI rule" rule="Host(`transmission.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=transmission@docker



time="2023-03-12T12:14:14Z" level=debug msg="Trying to challenge certificate for domain [ddns.xxx.yyy] found in HostSNI rule" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme rule="Host(`ddns.xxx.yyy`)" routerName=ddns-updater@docker

time="2023-03-12T12:14:14Z" level=debug msg="No ACME certificate generation required for domains [\"traefik.xxx.yyy\"]." rule="Host(`traefik.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=traefik@docker

time="2023-03-12T12:14:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"ddns.xxx.yyy\"]..." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme rule="Host(`ddns.xxx.yyy`)" routerName=ddns-updater@docker

time="2023-03-12T12:14:14Z" level=debug msg="No ACME certificate generation required for domains [\"ddns.xxx.yyy\"]." routerName=ddns-updater@docker ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme rule="Host(`ddns.xxx.yyy`)"

time="2023-03-12T12:14:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"home.xxx.yyy\"]..." rule="Host(`home.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=heimdall@docker

time="2023-03-12T12:14:14Z" level=debug msg="No ACME certificate generation required for domains [\"home.xxx.yyy\"]." providerName=myresolver.acme routerName=heimdall@docker rule="Host(`home.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory"

time="2023-03-12T12:14:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"transmission.xxx.yyy\"]..." rule="Host(`transmission.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=transmission@docker

time="2023-03-12T12:14:14Z" level=debug msg="No ACME certificate generation required for domains [\"transmission.xxx.yyy\"]." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=transmission@docker rule="Host(`transmission.xxx.yyy`)"

time="2023-03-12T12:14:14Z" level=debug msg="Looking for provided certificate(s) to validate [\"emby.xxx.yyy\"]..." routerName=emby@docker rule="Host(`emby.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme

time="2023-03-12T12:14:14Z" level=debug msg="No ACME certificate generation required for domains [\"emby.xxx.yyy\"]." rule="Host(`emby.xxx.yyy`)" ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=myresolver.acme routerName=emby@docker

time="2023-03-12T12:14:46Z" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 10.0.0.188:8096: i/o timeout"

My docker file is this:

version: "3.9"

services:
  traefik:
    image: "traefik:latest"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=ovh"
      - "--certificatesresolvers.myresolver.acme.email=A@B.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "8080:8080"
      - "443:443"
    environment:
      - "OVH_ENDPOINT=ovh-eu"
      - "OVH_APPLICATION_KEY=A"
      - "OVH_APPLICATION_SECRET=B"
      - "OVH_CONSUMER_KEY=C"
    volumes:
      - config-letsencrypt:/letsencrypt
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    restart: always
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.xxx.yyy`)"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls.certresolver=myresolver"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"
      - "traefik.http.middlewares.force-secure.redirectscheme.scheme=https"
      - "traefik.http.middlewares.force-secure.redirectscheme.permanent=true"
      - traefik.http.routers.http-catchall.rule=HostRegexp(`{any:.+}`)
      - traefik.http.routers.http-catchall.entrypoints=web
      - traefik.http.routers.http-catchall.middlewares=force-secure

  emby:
    image: lscr.io/linuxserver/emby:latest
    container_name: emby
    environment:
      - PUID=1028
      - PGID=100
      - TZ=Europe/Zurich
    volumes:
      - config-emby:/config
      - media:/data/
      # - media-movies:/data/movies
      #- /opt/vc/lib:/opt/vc/lib #optional
    ports:
      - 8096:8096
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.emby.rule=Host(`emby.xxx.yyy`)"
      - "traefik.http.routers.emby.entrypoints=websecure"
      - "traefik.http.routers.emby.tls.certresolver=myresolver"
      - "traefik.http.services.emby.loadbalancer.server.port=8096"
    restart: unless-stopped

  transmission:
    image: lscr.io/linuxserver/transmission:version-3.00-r8
    container_name: transmission
    environment:
      - PUID=1028
      - PGID=100
      - TZ=Europe/Zurich
    volumes:
      - config-transmission:/config
      - downloads:/downloads
      - movies:/movies
    ports:
      - 9091:9091
      - 51413:51413
      - 51413:51413/udp
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.transmission.rule=Host(`transmission.xxx.yyy`)"
      - "traefik.http.routers.transmission.entrypoints=websecure"
      - "traefik.http.routers.transmission.tls.certresolver=myresolver"
      - "traefik.http.services.transmission.loadbalancer.server.port=9091"

  medusa:
    image: lscr.io/linuxserver/medusa:latest
    container_name: medusa
    environment:
      - PUID=1028
      - PGID=100
      - TZ=Europe/London
    volumes:
      - config-medusa:/config
      - downloads:/downloads
      - tvshows:/tv
    ports:
      - 8081:8081
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.medusa.rule=Host(`medusa.xxx.yyy`)"
      - "traefik.http.routers.medusa.entrypoints=websecure"
      - "traefik.http.routers.medusa.tls.certresolver=myresolver"
      - "traefik.http.services.medusa.loadbalancer.server.port=8081"

  ddns-updater:
    image: qmcgaw/ddns-updater
    container_name: ddns-updater
    volumes:
      - config-ddns:/updater/data
    ports:
      - 8007:8007/tcp
    environment:
      - PERIOD=5m
      - CONFIG=
      - UPDATE_COOLDOWN_PERIOD=5m
      - PUBLICIP_FETCHERS=all
      - PUBLICIP_HTTP_PROVIDERS=all
      - PUBLICIPV4_HTTP_PROVIDERS=all
      - PUBLICIPV6_HTTP_PROVIDERS=all
      - PUBLICIP_DNS_PROVIDERS=all
      - PUBLICIP_DNS_TIMEOUT=3s
      - HTTP_TIMEOUT=10s

      # Web UI
      - LISTENING_PORT=8007
      - ROOT_URL=/

      # Backup
      - BACKUP_PERIOD=0 # 0 to disable
      - BACKUP_DIRECTORY=/updater/data

      # Other
      - LOG_LEVEL=info
      - LOG_CALLER=hidden
      - SHOUTRRR_ADDRESSES=
    restart: always
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.ddns-updater.rule=Host(`ddns.xxx.yyy`)"
      - "traefik.http.routers.ddns-updater.entrypoints=websecure"
      - "traefik.http.routers.ddns-updater.tls.certresolver=myresolver"
      - "traefik.http.services.ddns-updater.loadbalancer.server.port=8007"

  heimdall:
    image: lscr.io/linuxserver/heimdall:latest
    environment:
      - PUID=1028
      - PGID=100
      - TZ=Europe/London
    volumes:
      - config-heimdall:/config
    ports:
      - 444:443
    restart: unless-stopped
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.heimdall.rule=Host(`home.xxx.yyy`)"
      - "traefik.http.routers.heimdall.entrypoints=websecure"
      - "traefik.http.routers.heimdall.tls.certresolver=myresolver"
      - "traefik.http.routers.heimdall.tls=true"
      - "traefik.http.services.heimdall.loadbalancer.server.port=443" 
      - "traefik.http.services.heimdall.loadbalancer.server.scheme=https"

volumes:
  tvshows:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/media/tvshows"
  movies:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/media/movies"
  media:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/media"
  config-emby:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/emby"
  config-ddns:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/ddns"
  downloads:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/work/torrent/downloads"
  config-transmission:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/transmission"
  config-medusa:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/medusa"
  config-letsencrypt:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/letsencrypt"
  config-heimdall:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.0.60,nolock,rw,soft"
      device: ":/volume2/apps/config/heimdall"

I can totally reach my service(emby) on their exposed port if I don't get through the proxy. Yesterday it was working and I don't think I modified anything since then.

Any idea what could cause the issue?

2

Did you try to connect all containers to a Docker network? See this example.

3

My understanding is that a network is by default created for each docker-compose file? Also, I got it working yesterday for a while, but I'm really not sure what changed since then.

4

You are right. But you have to tell traefik to use this network. Try setting - --providers.docker.network=<your_network_name> in your traefik service command.

1 Like
5

Could this explain that sometimes it does work? Typically now, I've not connected to my portrainer instance ton configure anything since I started this post and I've full access?

6

Traefik will use all service IPs to forward traffic round robin. If Traefik only shares one network (of two) with the service, every other forward will fail.

If you set docker.network, Traefik will only use that network to forward traffic to. It can be set globally or per service.

7

I see, but the containers are only connected to one network, so if it uses round-robin to determine which network to use, only one would succeed, right? Also, as currently configured, traefik has also only access to one network.
I will try what you're indicating, seems to be a good things to do anyway, but I'm not sure it will really make a difference, I don't understand which other network it might try to use?

8

You are right, just every other case of gateway timeout here was about using multiple Docker networks :slight_smile:

You know that you don’t need to expose ports of your services, except for Traefik? Traefik will use the „internal“ ports within a Docker network anyway.

If you expose the ports, the services can be accessed from externally directly, going around any security middleware you have set up in Traefik.

9

You're totally correct. I keep them for the moment due to "suddenly the container doesn't respond but I don't know if it's the container, traefik or my computer". Once I'm sure everything is running smoothly, I will remove all of them except traefik ones.

But I still have issues(like this one: 400 Bad Request The plain HTTP request was sent to HTTPS port ) where I cannot access the container through traefik.

thanks for your help anyway, I really owe you a bunch of coffee once I finished configuring all this.

10

I love coffee, that sound very motivating :coffee: :slight_smile:

11

For others: it is good to denote the docker network for the traefik container to use (--providers.docker.network=docker_external) as mentioned above, but for large deployments it is also good to do the same for any containers that have more than one network (the problem I had specifically)

    labels:
      - traefik.enable=true
      - traefik.docker.network=docker_external
      - ...
4 Likes
12

You are an absolute legend. I've been trying to resolve this issue for so long; sometimes Docker would route my traffic correctly and sometimes it wouldn't regardless of whether the services ran just fine.

It turns out that Traefik was randomly deciding which network to route traffic through instead of always routing it through the external "web" network I setup because some services were a part of multiple networks. To make matters even more confusing, I was using hostnames like api.localhost for some of my services which comes with its own set of challenges.

Properly labeling each service with traefik.docker.network=web resolved the issue for me. Thank you.

3 Likes
13

Same for me. This fixed my issue. Thanks!!!

14

This issue was driving me crazy! At some point in time I wished I had gone back to nginx and get it done. Adding providers.docker.network=<external-net> on the treafik container solved the problem. All other containers used their own network and external-net network (created by traefik itself).

15

You can also set the default network in the static configuration:

providers:
  swarm:
    network: traefik

Like described in the documentation here: Traefik Documentation - Swarm - network

I use this with:

providers:
  swarm:
    exposedByDefault: false

So that only container I want to be exposed are picked up by traefik.