Middleware is sometimes not found when defined in the traefik service's docker labels

Traefik Traefik v2
,
3

Hi,
I was aware that a proper proof of concept would have been helpful but I have been trying to simplify my setup down to a reproducible anonymized example.

It really took me some time because on a clean (and more performant) test system it happens way less frequently. Not sure if adding more services actually ended up helping, but this is the exact configuration that triggered the behavior after some (~ 30) attempts:

version: '3'
name: example

services:
  traefik:
    image: traefik:v2.8
    restart: always
    command:
      - --api=true
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false
      - --entrypoints.websecure.address=:443
    labels:
      - traefik.enable=true

      - traefik.http.middlewares.default_ssl.headers.STSSeconds=3600

      - traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https_redirect.redirectscheme.port=50443
      - traefik.http.routers.https_redirect.rule=PathPrefix(`/`)
      - traefik.http.routers.https_redirect.entrypoints=websecure
      - traefik.http.routers.https_redirect.middlewares=https_redirect
    ports:
      - "443:443"
    volumes:
      - letsencrypt:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock
  whoamiA: # example service
    image: traefik/whoami
    restart: always
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.whoamiA.rule=Host(`whoamiA.example`)
      - traefik.http.routers.whoamiA.tls=true
      - traefik.http.routers.whoamiA.entrypoints=websecure
      - traefik.http.routers.whoamiA.middlewares=default_ssl
  whoamiB: # example service
    image: traefik/whoami
    restart: always
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.whoamiB.rule=Host(`whoamiB.example`)
      - traefik.http.routers.whoamiB.tls=true
      - traefik.http.routers.whoamiB.entrypoints=websecure
      - traefik.http.routers.whoamiB.middlewares=default_ssl
  whoamiC: # example service
    image: traefik/whoami
    restart: always
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.whoamiC.rule=Host(`whoamiC.example`)
      - traefik.http.routers.whoamiC.tls=true
      - traefik.http.routers.whoamiC.entrypoints=websecure
      - traefik.http.routers.whoamiC.middlewares=default_ssl
  whoamiD: # example service
    image: traefik/whoami
    restart: always
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.whoamiD.rule=Host(`whoamiD.example`)
      - traefik.http.routers.whoamiD.tls=true
      - traefik.http.routers.whoamiD.entrypoints=websecure
      - traefik.http.routers.whoamiD.middlewares=default_ssl
  whoamiE: # example service
    image: traefik/whoami
    restart: always
    labels:
      - com.centurylinklabs.watchtower.enable=true
      - traefik.enable=true
      - traefik.http.routers.whoamiE.rule=Host(`whoamiE.example`)
      - traefik.http.routers.whoamiE.tls=true
      - traefik.http.routers.whoamiE.entrypoints=websecure
      - traefik.http.routers.whoamiE.middlewares=default_ssl
volumes:
  letsencrypt:

I then repeatedly ran this:

docker compose stop traefik; docker compose rm -f traefik; docker compose up -d; docker compose logs -f traefik

In most of the cases, it works, i.e. it prints this:

example-traefik-1  | time="2022-09-30T20:22:32Z" level=info msg="Configuration loaded from flags."

If it prints that, just CTRL-C, Arrow-Up, Enter.
Keep repeating until you see something like this:

example-traefik-1  | time="2022-09-30T20:22:38Z" level=info msg="Configuration loaded from flags."
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" routerName=whoamiE@docker entryPointName=websecure
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" routerName=whoami@docker entryPointName=websecure
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" entryPointName=websecure routerName=whoamiA@docker
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" entryPointName=websecure routerName=whoamiB@docker
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" routerName=whoamiC@docker entryPointName=websecure
example-traefik-1  | time="2022-09-30T20:22:39Z" level=error msg="middleware \"default_ssl@docker\" does not exist" entryPointName=websecure routerName=whoamiD@docker