All services return 404

carlosoras · April 4, 2022, 7:08pm

Hello, first of all, thanks for your help, I'm quite new with docker and even newer with Traefik, so I'm sorry for all mistakes.

So, I was trying to install Traefik to expose some services, I just follow a tutorial and try to expose my first service, but I'm always getting a 404, even when I try to acces to dashboard.
I have a Raspberry with pihole installed, because of that I setted up port 82 in place of 80.
Also, in my router configuration, I'm routing the port 80 to my raspberry's ip, port 82, and I also redirected 443 port;

Here is my configuration, when I access to myIp:82/api I have a 404 page not found. Also if I try with my external ddns domain. and for all the services.
Also, my acme file is empty.
I tryied to enable the logs, but I'm not getting nothing in traefik.log, however in access.log I can see the querys.

docker-compose.yml

version: '3'

services:
  reverse-proxy:
    restart: always
    image: traefik:v2.6
    ports:
    - "443:443"
    - "82:82"
    volumes:
    - /home/pi/Traefik/traefik.toml:/etc/traefik/traefik.toml
    - /home/pi/Traefik/services.toml:/etc/traefik/services.toml
    - /var/run/docker.sock:/var/run/docker.sock
    - /home/pi/Traefik/acme.json:/acme.json
    labels:
    - "traefik.http.routers.api.rule=Host(`myawesomedomain.ddns.net`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"

    - "traefik.http.routers.api.service=api@internal"
    - "traefik.http.routers.api.entrypoints=http"
    - "traefik.http.routers.api.middlewares=auth"
    - "traefik.http.middlewares.auth.basicauth.users=pi:myencodedpassword"
    - "traefik.http.routers.transmission.entrypoints=http,https"
    - "traefik.http.routers.transmission.rule=Host(`myawesomedomain.ddns.net`) && (PathPrefix(`/transmission`)) "
    - "traefik.http.routers.transmission.service=transmission@file"
    - "traefik.http.routers.transmission.tls=true"
    - "traefik.http.routers.transmission.tls.certresolver=pi"

services.toml

[http]
  [http.services]
    [http.services.transmission]
      [http.services.transmission.loadBalancer]
        [[http.services.transmission.loadBalancer.servers]]
          url = "http://192.168.1.18:9091/"

traefik.toml

[entryPoints]
  [entryPoints.http]
  address = ":82"
  [entryPoints.https]
  address = ":443"
[traefikLog]
  filePath = "/var/log/traefik/traefik.log"
  level = "DEBUG"
[accessLog]
  filePath = "/var/log/traefik/access.log"
[api]
  dashboard = true
[providers.docker]
  endpoint = "unix:///var/run/docker.sock"
[providers.file]
  filename = "/etc/traefik/services.toml"
[certificatesResolvers.pi.acme]
  email = "mymail@mail.com"
  storage = "acme.json"
  [certificatesResolvers.pi.acme.httpChallenge]
    entryPoint = "http"

Thanks again

jakubhajek · April 6, 2022, 9:58am

Hello @carlosoras

Can you please share the log file while Traefik is starting? Especially the first few lines are important, when Traefik is loading the configuration and stating the appropriate provider.

carlosoras · April 6, 2022, 8:30pm

Thanks for your reply, so here is the logs

Creating network "traefik_default" with the default driver
Creating traefik_reverse-proxy_1 ... done
Attaching to traefik_reverse-proxy_1
reverse-proxy_1  | time="2022-04-06T20:00:45Z" level=info msg="Configuration loa                                                                             ded from file: /etc/traefik/traefik.toml"
reverse-proxy_1  | time="2022-04-06T20:00:45Z" level=error msg="The ACME resolve                                                                             r \"pi\" is skipped from the resolvers list because: unable to get ACME account:                                                                              permissions 644 for acme.json are too open, please use 600"
reverse-proxy_1  | time="2022-04-06T20:00:47Z" level=error msg="HTTP challenge i                                                                             s not enabled" entryPointName=http routerName=acme-http@internal
reverse-proxy_1  | time="2022-04-06T20:00:49Z" level=error msg="HTTP challenge i                                                                             s not enabled" entryPointName=http routerName=acme-http@internal
reverse-proxy_1  | time="2022-04-06T20:00:49Z" level=error msg="the router trans                                                                             mission@docker uses a non-existent resolver: pi"

After, I changed permissions for acme.json to 600, and I have this:

traefik_reverse-proxy_1 is up-to-date
Attaching to traefik_reverse-proxy_1
reverse-proxy_1  | time="2022-04-06T20:00:45Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.toml"
reverse-proxy_1  | time="2022-04-06T20:00:45Z" level=error msg="The ACME resolver \"pi\" is skipped from the resolvers list because: unable to get ACME account: permissions 644 for acme.json are too open, please use 600"
reverse-proxy_1  | time="2022-04-06T20:00:47Z" level=error msg="HTTP challenge is not enabled" entryPointName=http routerName=acme-http@internal
reverse-proxy_1  | time="2022-04-06T20:00:49Z" level=error msg="HTTP challenge is not enabled" entryPointName=http routerName=acme-http@internal
reverse-proxy_1  | time="2022-04-06T20:00:49Z" level=error msg="the router transmission@docker uses a non-existent resolver: pi"
reverse-proxy_1  | time="2022-04-06T20:02:53Z" level=error msg="accept tcp [::]:82: use of closed network connection" entryPointName=http
reverse-proxy_1  | time="2022-04-06T20:02:53Z" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=https
reverse-proxy_1  | time="2022-04-06T20:02:53Z" level=error msg="close tcp [::]:82: use of closed network connection" entryPointName=http
reverse-proxy_1  | time="2022-04-06T20:02:53Z" level=error msg="close tcp [::]:443: use of closed network connection" entryPointName=https

But now my acme.json is not empty anymore

carlosoras · April 10, 2022, 9:58am

Well, some updates. as I saw here https://community.traefik.io/t/2-3-errors-on-container-start-is-this-someting-to-worry-about/8438/8
using yml and toml can be a source of trouble, so I rewrite my files to yml.
But still I'm having this as logs:

Attaching to traefik_reverse-proxy_1
reverse-proxy_1  | time="2022-04-10T09:54:38Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
reverse-proxy_1  | time="2022-04-10T09:55:11Z" level=error msg="accept tcp [::]:82: use of closed network connection" entryPointName=http
reverse-proxy_1  | time="2022-04-10T09:55:11Z" level=error msg="Error while starting server: accept tcp [::]:82: use of closed network connection" entryPointName=http
reverse-proxy_1  | time="2022-04-10T09:55:11Z" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=https
reverse-proxy_1  | time="2022-04-10T09:55:11Z" level=error msg="Error while starting server: accept tcp [::]:443: use of closed network connection" entryPointName=https

peterhoneder · April 11, 2022, 4:35am

Hi!

what seems to happen here, is that your connections get closed. Traefik doesn't let you know why it closed the connections, but I would guess something is trying to stop traefik. If you run it, make sure nothing tries to stop or kill the traefik process or monitor if that could happen. Also one thing you could try it to turn on debug logs.

carlosoras · April 15, 2022, 4:36pm

I'm quite new to docker so I don't know what can be stopping traefik. I don't have other containers.
Do you have any clue how I can activate debug logs??
I try adding this in my traefik.yml:

log:
  level: DEBUG

But the log is still the same.

carlosoras · April 26, 2022, 7:55pm

well, after trying again, here is my log in debug, I don't see the closed network connection anymore, but still it doesn't work, any clues??

Attaching to traefik_reverse-proxy_1
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Traefik version 2.6.3 built on 2022-03-29T13:09:27Z"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"http\":{\"address\":\":82\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"udp\":{\"timeout\":\"3s\"}},\"https\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{},\"udp\":{\"timeout\":\"3s\"}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":\"15s\"},\"file\":{\"watch\":true,\"filename\":\"/etc/traefik/services.yml\"}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"accessLog\":{\"filePath\":\"/var/log/traefik/access.log\",\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"drop\"}}},\"certificatesResolvers\":{\"pi\":{\"acme\":{\"email\":\"mymail@mail.mail\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"http\"}}}},\"pilot\":{\"dashboard\":true}}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Start TCP Server" entryPointName=https
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider aggregator.ProviderAggregator"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Start TCP Server" entryPointName=http
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider *file.Provider"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="*file.Provider provider configuration: {\"watch\":true,\"filename\":\"/etc/traefik/services.yml\"}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider *traefik.Provider"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="*traefik.Provider provider configuration: {}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider *docker.Provider"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="*docker.Provider provider configuration: {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":\"15s\"}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider *acme.ChallengeTLSALPN"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="*acme.ChallengeTLSALPN provider configuration: {\"Timeout\":4000000000}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Starting provider *acme.Provider"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="*acme.Provider provider configuration: {\"email\":\"mymail@mail.mail\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"acme.json\",\"keyType\":\"RSA4096\",\"certificatesDuration\":2160,\"httpChallenge\":{\"entryPoint\":\"http\"},\"ResolverName\":\"pi\",\"store\":{},\"TLSChallengeProvider\":{\"Timeout\":4000000000},\"HTTPChallengeProvider\":{}}"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Attempt to renew certificates \"720h0m0s\" before expiry and check every \"24h0m0s\"" providerName=pi.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=info msg="Testing certificate renew..." providerName=pi.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Configuration received from provider file: {\"http\":{\"services\":{\"transmission\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://192.168.1.18:9091/\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{},\"tls\":{}}" providerName=file
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"routers\":{\"acme-http\":{\"entryPoints\":[\"http\"],\"service\":\"acme-http@internal\",\"rule\":\"PathPrefix(`/.well-known/acme-challenge/`)\",\"priority\":2147483647}},\"services\":{\"acme-http\":{},\"api\":{},\"dashboard\":{},\"noop\":{}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"tls\":{}}" providerName=internal
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Configuration received from provider pi.acme: {\"http\":{},\"tls\":{}}" providerName=pi.acme
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="No default certificate, generating one" tlsStoreName=default
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Provider connection established with docker 20.10.14 (API 1.41)" providerName=docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:09Z" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"api\":{\"entryPoints\":[\"http\",\"https\"],\"middlewares\":[\"auth\"],\"service\":\"api@internal\",\"rule\":\"Host(`myhost.com`) \\u0026\\u0026 (PathPrefix(`/api`) || PathPrefix(`/dashboard`))\"},\"transmission\":{\"entryPoints\":[\"http\",\"https\"],\"service\":\"transmission@file\",\"rule\":\"Host(`myhost.com`) \\u0026\\u0026 (PathPrefix(`/transmission`)) \",\"tls\":{\"certResolver\":\"pi\"}}},\"services\":{\"reverse-proxy-traefik\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.18.0.2:80\"}],\"passHostHeader\":true}}},\"middlewares\":{\"auth\":{\"basicAuth\":{\"users\":[\"pi:myencriptedpsw\"]}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:13Z" level=debug msg="Added outgoing tracing middleware acme-http@internal" entryPointName=http routerName=acme-http@internal middlewareName=tracing middlewareType=TracingForwarder
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:13Z" level=debug msg="Creating middleware" entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:13Z" level=debug msg="No default certificate, generating one" tlsStoreName=default
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:16Z" level=debug msg="Added outgoing tracing middleware acme-http@internal" routerName=acme-http@internal middlewareName=tracing middlewareType=TracingForwarder entryPointName=http
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:16Z" level=debug msg="Creating middleware" entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:16Z" level=debug msg="No default certificate, generating one" tlsStoreName=default
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:18Z" level=debug msg="Adding certificate for domain(s) myhost.com"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:18Z" level=debug msg="Added outgoing tracing middleware acme-http@internal" routerName=acme-http@internal entryPointName=http middlewareName=tracing middlewareType=TracingForwarder
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:18Z" level=debug msg="Creating middleware" entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:18Z" level=debug msg="No default certificate, generating one" tlsStoreName=default
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Adding certificate for domain(s) myhost.com"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Added outgoing tracing middleware api@internal" routerName=api@docker middlewareName=tracing middlewareType=TracingForwarder entryPointName=http
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" middlewareName=auth@docker middlewareType=BasicAuth entryPointName=http routerName=api@docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Adding tracing to middleware" entryPointName=http routerName=api@docker middlewareName=auth@docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Added outgoing tracing middleware acme-http@internal" entryPointName=http routerName=acme-http@internal middlewareType=TracingForwarder middlewareName=tracing
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=http
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" entryPointName=https middlewareName=traefik-internal-recovery middlewareType=Recovery
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" middlewareType=Pipelining entryPointName=http routerName=transmission@docker serviceName=transmission@file middlewareName=pipelining
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating load-balancer" entryPointName=http routerName=transmission@docker serviceName=transmission@file
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating server 0 http://192.168.1.18:9091/" entryPointName=http routerName=transmission@docker serviceName=transmission@file serverName=0
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="child http://192.168.1.18:9091/ now UP"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Propagating new UP status"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Added outgoing tracing middleware transmission@file" entryPointName=http middlewareName=tracing middlewareType=TracingForwarder routerName=transmission@docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" entryPointName=http middlewareName=traefik-internal-recovery middlewareType=Recovery
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=https
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Adding route for myhost.com with TLS options default" entryPointName=http
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Adding route for myhost.com with TLS options default" entryPointName=https
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Try to challenge certificate for domain [myhost.com] found in HostSNI rule" rule="Host(`myhost.com`) && (PathPrefix(`/transmission`)) " providerName=pi.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=transmission@docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="Looking for provided certificate(s) to validate [\"myhost.com\"]..." routerName=transmission@docker rule="Host(`myhost.com`) && (PathPrefix(`/transmission`)) " providerName=pi.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:20Z" level=debug msg="No ACME certificate generation required for domains [\"myhost.com\"]." rule="Host(`myhost.com`) && (PathPrefix(`/transmission`)) " providerName=pi.acme ACME CA="https://acme-v02.api.letsencrypt.org/directory" routerName=transmission@docker
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:42Z" level=debug msg="Serving default certificate for request: \"\""
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:42Z" level=debug msg="http: TLS handshake error from 192.168.1.6:62792: EOF"
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:42Z" level=debug msg="Serving default certificate for request: \"\""
e[36mreverse-proxy_1  |e[0m time="2022-04-26T19:47:49Z" level=debug msg="Serving default certificate for request: \"\""

Topic		Replies	Views
Traefik works great with one service, but a new one returns 404 Traefik v2 docker	10	1722	December 3, 2022
Receiving a 404 error only with external services Traefik v2 docker	0	1032	January 5, 2021
404 Error, not routing to external server Traefik v3 (latest) docker	2	231	August 23, 2024
Traefik Dashboard throws 404 error for /api routes Traefik v3 (latest) docker , dashboard-api	3	263	June 19, 2024
Getting 404 response despite seemingly correct router setup Traefik v3 (latest) docker , file	4	101	September 4, 2024

All services return 404

Related topics