Traefik doesn't seem to work with api@internal service

Hello,

I'm currently trying to get Traefik Dashboard in secure mode behind a middleware (Authelia in my case).
I however have some issues getting it to work. Here's my Traefik configuration:

version: '3.2'

networks:
  lbnet:
    driver: overlay
    attachable: true

services:
  traefik:
    image: traefik:v2.0
    command:
      - "--api.dashboard=true"
      - "--providers.docker.endpoint=unix:///var/run/docker.sock"
      - "--providers.docker.swarmMode=true"
      - "--providers.docker.exposedbydefault=true"
      - "--providers.docker.network=traefik_lbnet"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.letsencrypt.acme.dnsChallenge.resolvers=1.1.1.1:53"
      - "--certificatesresolvers.letsencrypt.acme.dnsChallenge.provider=ovh"
      - "--certificatesresolvers.letsencrypt.acme.email=mail@domain.com"
      - "--certificatesresolvers.letsencrypt.acme.storage=acme.json"
    ports:
      - "80:80"
      - "443:443"
      - target: 8080
        published: 8080
        mode: host
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /data/swarm-data/traefik/acme.json:/acme.json
    networks:
      - lbnet
    environment:
      - "OVH_ENDPOINT=ovh-eu"
      - "OVH_APPLICATION_KEY=REDACTED"
      - "OVH_APPLICATION_SECRET=REDACTED"
      - "OVH_CONSUMER_KEY=REDACTED"
    deploy:
      replicas: 2
      labels:
        - "traefik.http.routers.api.rule=Host(`traefik.dmz.internal.domain.com`)"
        - "traefik.http.routers.api.service=api@internal"
        - "traefik.http.routers.api.entrypoints=websecure"
        - "traefik.http.routers.api.tls.certresolver=letsencrypt"
        - "traefik.http.routers.api.middlewares=authelia-internal-io"
      placement:
        constraints: [node.role == manager]

I got it to work by tricking Traefik (enable api insecure, adding a service name docker, redirect service to api@docker) but I can't get it to work using api@internal without api.insecure.
Could you tell me if there's anything wrong with my configuration ?

Btw, great job on Traefik v2. Its new options are just amazing, dashboard is way better and useful than v1.7 and Proxy Protocol support is a bless behind an HAProxy in TCP mode...

Thanks for your help !

(Pinging @dduportal as requested by him)

Hi @Hakujou ,

as reference, you have a working example here, here and here.

When using Swarm Mode with the Traefik's provider "Docker", it is mandatory to specify the port that Traefik has to use for contacting the container with the label - "traefik.http.services.<service name>.loadbalancer.server.port=<port to use>".

Alas, even when using an internal service as api@internal which is never exposed outside Traefik's, you still have to specify a port that will be a "dummy" port.

Add this to the traefik's service labels and check again the result:

        - "traefik.http.services.api-svc.loadbalancer.server.port=9999 # The port value doesn't matter"

We are aware of this and are searching for a solution to improve the user experience, but it is definitively not a bug: https://github.com/containous/traefik/issues/5732 .

One of the steps, in short term, would be to provide a specific configuration example for Swarm in the case of the Dashboard, would it be a good thing for you?

Let us know

1 Like

Indeed, setting a dummy port make it works even with api@internal. It's definitely not ideal since it shouldn't be needed.

Also, yes, a Swarm specific example on documentation would be great. In general, some "User Guide" for Swarm would be great, like the one for 1.7. Some more documentation for common use cases (such as http to https redirection) are lacking in 2.0, and would be helpful.

Anyway, problem solved, which is great. Love the v2.0.

1 Like

Thanks. The pull requests https://github.com/containous/traefik/pull/5795 aims at giving a working example for enabling Traefik's Dashboard and API with Swarm, separately from Docker standalone. It should be a first step in the documentation improvement to avoid user being stuck as you were.

Just as a comment, I also needed the dummy port when running "network_mode: host" since I don't want to add any network configurations to my existing compose files but take advantage of traefik regardless).

version: "3.7"

services:

  tra:
    image: "traefik:latest"
    container_name: "traefik"
    restart: always
    command:
      - "--api.dashboard=true"
      - "--ping=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=true" #no config by default
      - "--providers.docker.defaultrule=Host(`{{ index .Labels \"com.docker.compose.service\"}}.$domain`)" #all services exposed as cname = service
      - "--entrypoints.web.address=:22280" #behind nginx until everything works
      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
      - "--entrypoints.web.http.redirections.entrypoint.permanent=true"
      - "--entrypoints.websecure.address=:22443"
      - "--entrypoints.websecure.http.tls.domains.main=$domain"
      - "--entrypoints.traefik.address=:22288" #no conflict for dashboard port
      - "--accesslog=true"
      - "--accesslog.filepath=/etc/traefik/access.log"
      - "--log.filePath=/etc/traefik/traefik.log"
      - "--providers.file.filename=/etc/traefik/dynamic.yaml" #read tls config and manual routes
      - "--providers.file.watch=true"
    labels:
      - "traefik.http.routers.api.service=api@internal" #dashboard is internal
      - "traefik.http.services.api-svc.loadbalancer.server.port=22288" # ! Important without this, the dashboard will return 404
      - "traefik.http.middlewares.sso.forwardauth.address=https://sso.$domain/_oauth"
      - "traefik.http.middlewares.sso.forwardauth.trustForwardHeader=true"
      - "traefik.http.middlewares.sso.forwardauth.authResponseHeaders=X-Auth-User"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./:/etc/traefik"
    network_mode: host

To add to the problem of dummy port for api@internal, I have not found a configuration where the Traefik would accept dynamic .yml definition of used certificate for api@internal. Letsencrypt works though seldom it is so that Traefik dashboard would be exposed to public. It might work with static cert provider (in docker.yml) pointing to certificate files directly, have not tried.

Ps. this problem still exists 3 years later.

Check simple Traefik example.

Got it working. Own mistake of 2x file provider in Traefik config. Silly me!