Can not reproduce Traefik quick start. Getting http 503 error

Twofyw · May 13, 2023, 10:56am

I'm following Traefik quick start and arrived at the following docker compose file:

version: '3'

services:
  traefik:
    image: traefik:v2.10
    restart: always
    command: --api.insecure=true --providers.docker
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  whoami:
    image: traefik/whoami
    labels:
      - "traefik.http.routers.whoami.rule=Host(`whoami.docker.localhost`)"

However it always give me http 503 error.

$ curl -v -H Host:whoami.docker.localhost http://127.0.0.1
*   Trying 127.0.0.1:80...
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host:whoami.docker.localhost
> User-Agent: curl/8.0.1
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Content-Length: 0
< Date: Sat, 13 May 2023 10:43:50 GMT
<
* Connection #0 to host 127.0.0.1 left intact

I've tried to replace the service with python -m http.server using static config and it works. So I wonder if the 503 error is related to the whoami image.

bluepuma77 · May 13, 2023, 1:23pm

Try setting --providers.docker=true.

Enable Traefik debug log and access log to see what's happening.

Personally I like it better to have a dedicated Docker network (like proxy) to connect the containers, not the default bridge network, check this simple Traefik example.

Twofyw · May 13, 2023, 1:59pm

Thanks for your ideas of enabling logging and using a dedicated network. I've added them to the docker compose file:

version: '3.9'

services:
  traefik:
    image: traefik:v2.10
    networks:
      - proxy
    restart: always
    command: --api.insecure=true --providers.docker --providers.docker.network=proxy --log.level=DEBUG --log.filePath=/logs/log.log --accessLog.filePath=/logs/access.log
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./logs:/logs
  whoami:
    image: traefik/whoami:v1.8
    networks:
      - proxy
    labels:
      - "traefik.docker.network=proxy"
      - "traefik.http.routers.whoami.rule=Host(`whoami.docker.localhost`)"

networks:
  proxy:
    name: proxy

Log content:
access.log:

172.28.0.1 - - [13/May/2023:13:52:18 +0000] "GET /api/overview HTTP/1.1" 200 497 "-" "-" 1 "api@internal" "-" 0ms
172.28.0.1 - - [13/May/2023:13:52:21 +0000] "GET / HTTP/1.1" 503 0 "-" "-" 2 "whoami@docker" "http://172.28.0.2:80" 1442ms
172.28.0.1 - - [13/May/2023:13:52:23 +0000] "GET /api/overview HTTP/1.1" 200 497 "-" "-" 3 "api@internal" "-" 0ms

log.log:

time="2023-05-13T13:52:17Z" level=info msg="Traefik version 2.10.1 built on 2023-04-27T14:52:35Z"
time="2023-05-13T13:52:17Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}},\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"http2\":{\"maxConcurrentStreams\":250},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"network\":\"proxy\",\"swarmModeRefreshSeconds\":\"15s\"}},\"api\":{\"insecure\":true,\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"filePath\":\"/logs/log.log\",\"format\":\"common\"},\"accessLog\":{\"filePath\":\"/logs/access.log\",\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"drop\"}}}}"
time="2023-05-13T13:52:17Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
time="2023-05-13T13:52:17Z" level=info msg="Starting provider aggregator aggregator.ProviderAggregator"
time="2023-05-13T13:52:17Z" level=debug msg="Starting TCP Server" entryPointName=traefik
time="2023-05-13T13:52:17Z" level=debug msg="Starting TCP Server" entryPointName=http
time="2023-05-13T13:52:17Z" level=info msg="Starting provider *docker.Provider"
time="2023-05-13T13:52:17Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
time="2023-05-13T13:52:17Z" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"network\":\"proxy\",\"swarmModeRefreshSeconds\":\"15s\"}"
time="2023-05-13T13:52:17Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {}"
time="2023-05-13T13:52:17Z" level=info msg="Starting provider *traefik.Provider"
time="2023-05-13T13:52:17Z" level=debug msg="*traefik.Provider provider configuration: {}"
time="2023-05-13T13:52:17Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"traefik\"],\"service\":\"api@internal\",\"rule\":\"PathPrefix(`/api`)\",\"priority\":2147483646},\"dashboard\":{\"entryPoints\":[\"traefik\"],\"middlewares\":[\"dashboard_redirect@internal\",\"dashboard_stripprefix@internal\"],\"service\":\"dashboard@internal\",\"rule\":\"PathPrefix(`/`)\",\"priority\":2147483645}},\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"middlewares\":{\"dashboard_redirect\":{\"redirectRegex\":{\"regex\":\"^(http:\\\\/\\\\/(\\\\[[\\\\w:.]+\\\\]|[\\\\w\\\\._-]+)(:\\\\d+)?)\\\\/$\",\"replacement\":\"${1}/dashboard/\",\"permanent\":true}},\"dashboard_stripprefix\":{\"stripPrefix\":{\"prefixes\":[\"/dashboard/\",\"/dashboard\"]}}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=internal
time="2023-05-13T13:52:17Z" level=debug msg="Provider connection established with docker 23.0.6 (API 1.42)" providerName=docker
time="2023-05-13T13:52:17Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"traefik-traefik\":{\"service\":\"traefik-traefik\",\"rule\":\"Host(`traefik-traefik`)\"},\"whoami\":{\"service\":\"whoami-traefik\",\"rule\":\"Host(`whoami.docker.localhost`)\"}},\"services\":{\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.28.0.3:80\"}],\"passHostHeader\":true}},\"whoami-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.28.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-05-13T13:52:17Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" entryPointName=traefik routerName=dashboard@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareType=StripPrefix entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal
time="2023-05-13T13:52:17Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_stripprefix@internal entryPointName=traefik routerName=dashboard@internal
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-05-13T13:52:17Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik
time="2023-05-13T13:52:17Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware api@internal" entryPointName=traefik routerName=api@internal middlewareName=tracing middlewareType=TracingForwarder
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=traefik
time="2023-05-13T13:52:17Z" level=debug msg="No entryPoint defined for this router, using the default one(s) instead: [http]" routerName=traefik-traefik
time="2023-05-13T13:52:17Z" level=debug msg="No entryPoint defined for this router, using the default one(s) instead: [http]" routerName=whoami
time="2023-05-13T13:52:17Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=api@internal
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=dashboard@internal
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareType=StripPrefix entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal
time="2023-05-13T13:52:17Z" level=debug msg="Adding tracing to middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_stripprefix@internal
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex entryPointName=traefik routerName=dashboard@internal
time="2023-05-13T13:52:17Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-05-13T13:52:17Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_redirect@internal entryPointName=traefik
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=traefik
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareType=Pipelining serviceName=traefik-traefik entryPointName=http routerName=traefik-traefik@docker middlewareName=pipelining
time="2023-05-13T13:52:17Z" level=debug msg="Creating load-balancer" entryPointName=http routerName=traefik-traefik@docker serviceName=traefik-traefik
time="2023-05-13T13:52:17Z" level=debug msg="Creating server 0 http://172.28.0.3:80" routerName=traefik-traefik@docker serviceName=traefik-traefik entryPointName=http serverName=0
time="2023-05-13T13:52:17Z" level=debug msg="child http://172.28.0.3:80 now UP"
time="2023-05-13T13:52:17Z" level=debug msg="Propagating new UP status"
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware traefik-traefik" middlewareType=TracingForwarder entryPointName=http routerName=traefik-traefik@docker middlewareName=tracing
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" routerName=whoami@docker serviceName=whoami-traefik entryPointName=http middlewareName=pipelining middlewareType=Pipelining
time="2023-05-13T13:52:17Z" level=debug msg="Creating load-balancer" serviceName=whoami-traefik entryPointName=http routerName=whoami@docker
time="2023-05-13T13:52:17Z" level=debug msg="Creating server 0 http://172.28.0.2:80" routerName=whoami@docker serviceName=whoami-traefik serverName=0 entryPointName=http
time="2023-05-13T13:52:17Z" level=debug msg="child http://172.28.0.2:80 now UP"
time="2023-05-13T13:52:17Z" level=debug msg="Propagating new UP status"
time="2023-05-13T13:52:17Z" level=debug msg="Added outgoing tracing middleware whoami-traefik" routerName=whoami@docker middlewareName=tracing middlewareType=TracingForwarder entryPointName=http
time="2023-05-13T13:52:17Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=http middlewareName=traefik-internal-recovery
time="2023-05-13T13:52:23Z" level=info msg="I have to go..."
time="2023-05-13T13:52:23Z" level=info msg="Stopping server gracefully"
time="2023-05-13T13:52:23Z" level=debug msg="Waiting 10s seconds before killing connections." entryPointName=http
time="2023-05-13T13:52:23Z" level=debug msg="Waiting 10s seconds before killing connections." entryPointName=traefik
time="2023-05-13T13:52:23Z" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2023-05-13T13:52:23Z" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2023-05-13T13:52:23Z" level=error msg="accept tcp [::]:8080: use of closed network connection" entryPointName=traefik
time="2023-05-13T13:52:23Z" level=error msg="close tcp [::]:8080: use of closed network connection" entryPointName=traefik
time="2023-05-13T13:52:23Z" level=debug msg="Entry point http closed" entryPointName=http
time="2023-05-13T13:52:24Z" level=debug msg="Provider event received {Status:die ID:a3ca26a491fbddba89fe8e19b22eb3b913d58d031b8b8c787a8c35d89ac98991 From:traefik/whoami:v1.8 Type:container Action:die Actor:{ID:a3ca26a491fbddba89fe8e19b22eb3b913d58d031b8b8c787a8c35d89ac98991 Attributes:map[com.docker.compose.config-hash:a9fb8bd5f2b83e3c28f883d924f01e504dc999351fe7a24d978c5c74eeda79a0 com.docker.compose.container-number:1 com.docker.compose.depends_on: com.docker.compose.image:sha256:420d3c2af3570ad1ced363bce5b284a4c662cf7ee178e261ae42b7d750d488a4 com.docker.compose.oneoff:False com.docker.compose.project:traefik com.docker.compose.project.config_files:/home/twofyw/projects/docker-compose/traefik/docker-compose-example.yaml com.docker.compose.project.working_dir:/home/twofyw/projects/docker-compose/traefik com.docker.compose.replace:0c07bc6cd48a2a0d14b5c7aa7acd6a1e579651030ca79982eefb7da5b30ba711 com.docker.compose.service:whoami com.docker.compose.version:2.17.3 exitCode:2 image:traefik/whoami:v1.8 name:traefik-whoami-1 org.opencontainers.image.created:2022-10-15T13:46:19Z org.opencontainers.image.description:Tiny Go webserver that prints OS information and HTTP request to output org.opencontainers.image.revision:e59736736851ca955cf8734fe4ea6d0c71ddbd62 org.opencontainers.image.source:https://github.com/traefik/whoami org.opencontainers.image.title:whoami org.opencontainers.image.version:1.8.7 traefik.docker.network:proxy traefik.http.routers.whoami.rule:Host(`whoami.docker.localhost`)]} Scope:local Time:1683985944 TimeNano:1683985944160486641}" providerName=docker
time="2023-05-13T13:52:24Z" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"traefik-traefik\":{\"service\":\"traefik-traefik\",\"rule\":\"Host(`traefik-traefik`)\"}},\"services\":{\"traefik-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.28.0.3:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
time="2023-05-13T13:52:24Z" level=debug msg="No entryPoint defined for this router, using the default one(s) instead: [http]" routerName=traefik-traefik
time="2023-05-13T13:52:24Z" level=debug msg="No default certificate, fallback to the internal generated certificate" tlsStoreName=default
time="2023-05-13T13:52:24Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik routerName=api@internal
time="2023-05-13T13:52:24Z" level=debug msg="Added outgoing tracing middleware dashboard@internal" middlewareName=tracing entryPointName=traefik routerName=dashboard@internal middlewareType=TracingForwarder
time="2023-05-13T13:52:24Z" level=debug msg="Creating middleware" middlewareType=StripPrefix middlewareName=dashboard_stripprefix@internal entryPointName=traefik routerName=dashboard@internal
time="2023-05-13T13:52:24Z" level=debug msg="Adding tracing to middleware" middlewareName=dashboard_stripprefix@internal entryPointName=traefik routerName=dashboard@internal
time="2023-05-13T13:52:24Z" level=debug msg="Creating middleware" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-05-13T13:52:24Z" level=debug msg="Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/" entryPointName=traefik routerName=dashboard@internal middlewareName=dashboard_redirect@internal middlewareType=RedirectRegex
time="2023-05-13T13:52:24Z" level=debug msg="Adding tracing to middleware" routerName=dashboard@internal middlewareName=dashboard_redirect@internal entryPointName=traefik
time="2023-05-13T13:52:24Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2023-05-13T13:52:24Z" level=debug msg="Creating middleware" middlewareName=pipelining middlewareType=Pipelining routerName=traefik-traefik@docker entryPointName=http serviceName=traefik-traefik
time="2023-05-13T13:52:24Z" level=debug msg="Creating load-balancer" entryPointName=http serviceName=traefik-traefik routerName=traefik-traefik@docker
time="2023-05-13T13:52:24Z" level=debug msg="Creating server 0 http://172.28.0.3:80" entryPointName=http serverName=0 serviceName=traefik-traefik routerName=traefik-traefik@docker
time="2023-05-13T13:52:24Z" level=debug msg="child http://172.28.0.3:80 now UP"
time="2023-05-13T13:52:24Z" level=debug msg="Propagating new UP status"
time="2023-05-13T13:52:24Z" level=debug msg="Added outgoing tracing middleware traefik-traefik" entryPointName=http routerName=traefik-traefik@docker middlewareName=tracing middlewareType=TracingForwarder
time="2023-05-13T13:52:24Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=http
time="2023-05-13T13:52:24Z" level=debug msg="Entry point traefik closed" entryPointName=traefik
time="2023-05-13T13:52:24Z" level=info msg="Server stopped"
time="2023-05-13T13:52:24Z" level=info msg="Shutting down"

It looks like a problem is the whoami server not responding to traefik's request. However the whoami server is accessible from the host (the ip address is different because it from a different run. I forgot to copy the log from the first run):

$ curl http://172.28.0.3:80
Hostname: a3ca26a491fb
IP: 127.0.0.1
IP: 172.28.0.3
RemoteAddr: 172.28.0.1:37748
GET / HTTP/1.1
Host: 172.28.0.3
User-Agent: curl/8.0.1
Accept: */*

Twofyw · May 13, 2023, 2:10pm

I found the problem. I set http proxy in ~/.docker/config.json recently and totally forgot about it, until I ran wget http://172.28.0.3:80 in the traefik's container:

Connecting to 192.168.1.4:8118 (192.168.1.4:8118)
wget: server returned error: HTTP/1.1 503 Service Unavailable

At least I learned that traefik honors http proxy environment variables

system · May 16, 2023, 2:11pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
404 not found but everything else looks fine with Traefik v2 Traefik v2 (latest) docker	4	533	September 28, 2019
Traefik: Configuration with docker not working Traefik v2 (latest) docker , dashboard-api	2	2414	May 28, 2020
Requests with https lead to 404, traefik v2.10 and a simple docker-compose file Traefik v2 (latest) docker	3	270	November 21, 2023
Followed "quick start" but http "whoami" returns 404, what is wrong ? [help needed] [in progress] Traefik v2 (latest) docker , cli	3	2379	December 31, 2019
503 issue, server and router up Traefik v2 (latest) docker , file , middleware	10	264	July 19, 2023

Can not reproduce Traefik quick start. Getting http 503 error

Related Topics