Problem with Traefik `Not Found: HTTP status code 404;`

Why I get 404 status?

my docker service:

user:
  image: userservice
  labels:
    - traefik.enable=true
    - traefik.http.routers.user.rule=PathPrefix(`/proto.UserService/`)
    - traefik.http.services.user.loadBalancer.server.scheme=h2c
    - traefik.http.services.user.loadBalancer.server.port=10101

Hello @batazor,

Thanks for your interest Traefik!

I think the problem is that you have missed to attach the configured service to your router. The labels on your userservice should be:

user:
  image: userservice
  labels:
    - traefik.enable=true
    - traefik.http.routers.user.rule=PathPrefix(`/proto.UserService/`)
    - traefik.http.routers.user.service=user

    - traefik.http.services.user.loadBalancer.server.scheme=h2c
    - traefik.http.services.user.loadBalancer.server.port=10101

Check out the following documentation for more details.

Hope this helps.

No, it did not solve my problem, the error remained the same.
Usually, if there is an error in finding service then traefik will return the error code 502.

It seems to me that the problem is in the connection of my service with the traefik itself. The service worked correctly with traefik 1.7.

Can you share your static configuration and the complete Traefik logs (from the beginning)?

If you have a 404 response maybe:

  • the request is not matching the router rule
  • the router does not exist because of a configuration error

Yes. @kevinpollet

My traefik.toml

[global]
  checkNewVersion = true
  sendAnonymousUsage = true

[entryPoints]
  [entryPoints.web]
    address = ":80"

  [entryPoints.websecure]
    address = ":443"

  [entryPoints.metrics]
    address = ":8082"

[log]
  level = "DEBUG"
  format = "json"

[api]
  insecure = true
  dashboard = true

[ping]

[providers.file]
  directory = "/conf"
  watch = true

[providers.docker]
  endpoint = "unix:///var/run/docker.sock"

  swarmMode = true
  swarmModeRefreshSeconds = 10

  exposedByDefault = false
  watch = true
  network = "net"

[tracing]
  serviceName = "traefik"
  spanNameLimit = 0
  [tracing.jaeger]
    samplingServerURL = "http://jaeger-agent:5778/sampling"
    samplingType = "const"
    samplingParam = 1.0
    localAgentHostPort = "jaeger-agent:6831"
    traceContextHeaderName = "uber-trace-id"

[metrics]
  [metrics.prometheus]
    addEntryPointsLabels = true
    addServicesLabels = true
    entryPoint = "metrics"

my dynamic_conf.toml

[tls]
  [[tls.certificates]]
    certFile = "/cert/backend.cert"
    keyFile = "/cert/backend.key"
    stores = ["default"]
  [tls.stores]
    [tls.stores.default]
      [tls.stores.default.defaultCertificate]
        certFile = "/cert/backend.cert"
        keyFile = "/cert/backend.key"

and docker-compose.yml

version: '3.7'

services:

 traefik:
    image: traefik:v2.3
    command:
      - --providers.docker.network=net
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints: [ node.role == manager ]
      restart_policy:
        condition: on-failure
    ports:
      - "443:443"
      - "8060:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /traefik/cert:/cert
      - /traefik/traefik.toml:/traefik.toml
      - /traefik/dynamic_conf.toml:/conf/dynamic_conf.toml
    networks:
      - net

  user:
    image: user:latest
    deploy:
      labels:
        - traefik.enable=true
        - traefik.docker.network=net
        - traefik.http.routers.user.rule=PathPrefix(`/proto.UserService/`)
        - traefik.http.services.user.loadBalancer.server.scheme=h2c
        - traefik.http.routers.user.service=user
        - traefik.http.services.user.loadBalancer.server.port=10101
      mode: replicated
      replicas: 1
    networks:
      - net

And traefik logs

{"entryPointName":"traefik","level":"debug","middlewareName":"dashboard_stripprefix@internal","msg":"Adding tracing to middleware","routerName":"dashboard@internal","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"traefik","level":"debug","middlewareName":"dashboard_redirect@internal","middlewareType":"RedirectRegex","msg":"Creating middleware","routerName":"dashboard@internal","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"traefik","level":"debug","middlewareName":"dashboard_redirect@internal","middlewareType":"RedirectRegex","msg":"Setting up redirection from ^(http:\\/\\/(\\[[\\w:.]+\\]|[\\w\\._-]+)(:\\d+)?)\\/$ to ${1}/dashboard/","routerName":"dashboard@internal","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"traefik","level":"debug","middlewareName":"dashboard_redirect@internal","msg":"Adding tracing to middleware","routerName":"dashboard@internal","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"traefik","level":"debug","middlewareName":"traefik-internal-recovery","middlewareType":"Recovery","msg":"Creating middleware","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"metrics","level":"debug","middlewareName":"metrics-entrypoint","middlewareType":"Metrics","msg":"Creating middleware","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"metrics","level":"debug","middlewareName":"tracing","middlewareType":"TracingEntryPoint","msg":"Creating middleware","time":"2020-12-03T18:22:15Z"}


{"entryPointName":"web","level":"debug","middlewareName":"metrics-entrypoint","middlewareType":"Metrics","msg":"Creating middleware","time":"2020-12-03T18:22:15Z"}

Hi @batazor

With swarm mode the labels have to be nested under the deploy key, you did this with the traefik service but not with your user service

Version - traefik.http.routers.user.service=user@docker did not change the situation.

Or is it something else?

Sorry I was looking at your original post where you had:

  image: userservice
  labels:
    - traefik.enable=true
    - traefik.http.routers.user.rule=PathPrefix(`/proto.UserService/`)
    - traefik.http.services.user.loadBalancer.server.scheme=h2c
    - traefik.http.services.user.loadBalancer.server.port=10101

I now see you posted an updated one later.

user:
    image: user:latest
    deploy:
      labels:
....

So, are there any other ideas how I can find the problem?

Hi @batazor

When looking at routing issues I always ensure that the accesslog is enable(I prefer json format for verbosity).

As well as logging the request the upstream router is logged(or not) if it matches. I have had the experience of the actual backend service returning 404, but judicious use of headers in your services should make that easy to determine anyway.