Putting labels on services in Docker swarm doesn't work

Traefik Traefik v2
1

Documentation for the docker provider instructs that in swarm mode, labels should be attached to the service under the deploy section.

For all service deployments, I'm using the command docker stack deploy -c docker-compose.yml nublab

First, I deployed traefik, with the following file:

---
version: '3.3'

services:
  traefik:
    image: traefik
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    volumes:
      - /var/lib/traefik/config:/config
      - /var/lib/traefik/config/traefik.toml:/etc/traefik/traefik.toml
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - traefik
    secrets:
      - test

secrets:
  test:
    external: true

networks:
      traefik:

I can access the dashboard just fine.

Then I deployed heimdall with this config:

---
version: "3.3"

services:
  heimdall:
    image: linuxserver/heimdall
    container_name: heimdall
    volumes:
      - /var/lib/heimdall/config:/config
    restart: unless-stopped
    deploy:
      labels:
          com.centurylinklabs.watchtower.enable: "true"
          traefik.enable: "true"
          traefik.http.routers.heimdall.entrypoints: "websecure"
          traefik.http.routers.heimdall.tls: "true"
          traefik.subdomain: "start"

For some reason traefik did not pickup the new service. So then I tried the old style of labels, and redeployed, and it works.

---
version: "3.3"

services:
  heimdall:
    image: linuxserver/heimdall
    container_name: heimdall
    volumes:
      - /var/lib/heimdall/config:/config
    restart: unless-stopped
    labels:
        com.centurylinklabs.watchtower.enable: "true"
        traefik.enable: "true"
        traefik.http.routers.heimdall.entrypoints: "websecure"
        traefik.http.routers.heimdall.tls: "true"
        traefik.subdomain: "start"

Looking at the logs, it appears that in the first case, while traefik does pick up the new service, it does not see the labels when they're under deploy

Output of traefik version: (What version of Traefik are you using?)

Version:      2.0.1
Codename:     montdor
Go version:   go1.13.1
Built:        2019-09-26T16:18:03Z
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

################################################################
# Global configuration
################################################################
[global]
  checkNewVersion = true
  sendAnonymousUsage = true

################################################################
# Entrypoints configuration
################################################################

# Entrypoints definition
#
# Optional
# Default:
[entryPoints]
  [entryPoints.web]
    address = ":80"

  [entryPoints.websecure]
    address = ":443"

  [entryPoints.traefik]
    address = ":8080"

################################################################
# Redirect HTTP to HTTPS
################################################################

[http.routers]
  [http.routers.https-redirect]
  rule = "HostRegexp(`{any:.*}`)"
  middlewares = ["https-redirect"]
  service = "dummy"

[http.middlewares]
  [http.middlewares.https-redirect.redirectscheme]
    scheme = "https"

[http.services]
  [http.services.dummy.LoadBalancer]
     [[http.services.dummy.LoadBalancer.servers]]
        url = ""

################################################################
# Traefik logs configuration
################################################################

# Traefik logs
# Enabled by default and log to stdout
#
# Optional
#
[log]

  # Log level
  #
  # Optional
  # Default: "ERROR"
  #
  level = "DEBUG"

  # Sets the filepath for the traefik log. If not specified, stdout will be used.
  # Intermediate directories are created if necessary.
  #
  # Optional
  # Default: os.Stdout
  #
  # filePath = "log/traefik.log"

  # Format is either "json" or "common".
  #
  # Optional
  # Default: "common"
  #
  # format = "json"

################################################################
# Access logs configuration
################################################################

# Enable access logs
# By default it will write to stdout and produce logs in the textual
# Common Log Format (CLF), extended with additional fields.
#
# Optional
#
# [accessLog]

  # Sets the file path for the access log. If not specified, stdout will be used.
  # Intermediate directories are created if necessary.
  #
  # Optional
  # Default: os.Stdout
  #
  # filePath = "/path/to/log/log.txt"

  # Format is either "json" or "common".
  #
  # Optional
  # Default: "common"
  #
  # format = "json"

################################################################
# API and dashboard configuration
################################################################

# Enable API and dashboard
[api]
  dashboard = true
  insecure = true
  # Name of the related entry point
  #
  # Optional
  # Default: "traefik"
  #
  # entryPoint = "traefik"

  # Enabled Dashboard
  #
  # Optional
  # Default: true
  #
  # dashboard = false

################################################################
# Ping configuration
################################################################

# Enable ping
[ping]

  # Name of the related entry point
  #
  # Optional
  # Default: "traefik"
  #
  # entryPoint = "traefik"

################################################################
# Docker configuration backend
################################################################

# Enable Docker configuration backend
[providers.docker]

  # Docker server endpoint. Can be a tcp or a unix socket endpoint.
  #
  # Required
  # Default: "unix:///var/run/docker.sock"
  #
 endpoint = "unix:///var/run/docker.sock"
# endpoint = "tcp://127.0.0.1:2375"

  # Default host rule.
  #
  # Optional
  # Default: "Host(`{{ normalize .Name }}`)"
  #
  defaultRule = "Host(`{{ index .Labels \"traefik.subdomain\" }}.testserver.nub.local`)"

  # Expose containers by default in traefik
  #
  # Optional
  # Default: true
  #
  exposedByDefault = false


################################################################
# File configuration backend
################################################################

#[providers.file]

################################################################
# ACME Configuration
################################################################

[certificatesResolvers.LE.acme]

  # Email address used for registration.
  #
  # Required
  #
  email = "test@test.com"

  # File or key used for certificates storage.
  #
  # Required
  #
  storage = "/config/acme.json"

  # CA server to use.
  # Uncomment the line to use Let's Encrypt's staging server,
  # leave commented to go to prod.
  #
  # Optional
  # Default: "https://acme-v02.api.letsencrypt.org/directory"
  #
  # caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"

  # KeyType to use.
  #
  # Optional
  # Default: "RSA4096"
  #
  # Available values : "EC256", "EC384", "RSA2048", "RSA4096", "RSA8192"
  #
  # keyType = "RSA4096"

  # Use a TLS-ALPN-01 ACME challenge.
  #
  # Optional (but recommended)
  #
  # [certificatesResolvers.LE.acme.tlsChallenge]

  # Use a HTTP-01 ACME challenge.
  #
  # Optional
  #
  # [certificatesResolvers.LE.acme.httpChallenge]

    # EntryPoint to use for the HTTP-01 challenges.
    #
    # Required
    #
    # entryPoint = "web"

  # Use a DNS-01 ACME challenge rather than HTTP-01 challenge.
  # Note: mandatory for wildcard certificate generation.
  #
  # Optional
  #
  #[certificatesResolvers.LE.acme.dnsChallenge]

    # DNS provider used.
    #
    # Required
    #
   # provider = "{{ acme_provider }}"

    # By default, the provider will verify the TXT DNS challenge record before letting ACME verify.
    # If delayBeforeCheck is greater than zero, this check is delayed for the configured duration in seconds.
    # Useful if internal networks block external DNS queries.
    #
    # Optional
    # Default: 0
    #
    # delayBeforeCheck = 0

    # Use following DNS servers to resolve the FQDN authority.
    #
    # Optional
    # Default: empty
    #
    # resolvers = ["1.1.1.1:53", "8.8.8.8:53"]

    # Disable the DNS propagation checks before notifying ACME that the DNS challenge is ready.
    #
    # NOT RECOMMENDED:
    # Increase the risk of reaching Let's Encrypt's rate limits.
    #
    # Optional
    # Default: false
    #
    # disablePropagationCheck = true

################################################################
# TLS Configuration
################################################################

[tls.options]
  [tls.options.default]
    minVersion = "VersionTSL12"
    cipherSuites = [
      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
      "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
      "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
      "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
      "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
      "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
      "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
      "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
      "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
      "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
      "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
      "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
      "TLS_DHE_RSA_WITH_AES_128_CBC_SHA",
      "TLS_DHE_RSA_WITH_AES_256_CBC_SHA",
      "TLS_DHE_RSA_WITH_AES_128_CBC_SHA256",
      "TLS_DHE_RSA_WITH_AES_256_CBC_SHA256"
    ]

If applicable, please paste the log output in DEBUG level (--log.level=DEBUG switch)

nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.toml"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Traefik version 2.0.1 built on 2019-09-26T16:18:03Z"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true,\"sendAnonymousUsage\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ index .Labels \\\"traefik.subdomain\\\" }}.testserver.nub.local`)\",\"swarmModeRefreshSeconds\":15000000000}},\"api\":{\"insecure\":true,\"dashboard\":true},\"ping\":{\"entryPoint\":\"traefik\"},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"LE\":{\"acme\":{\"email\":\"test@test.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"/config/acme.json\",\"keyType\":\"RSA4096\"}}}}"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Stats collection is enabled."
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Many thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration."
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Help us improve Traefik by leaving this feature on :)"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="More details on: https://docs.traefik.io/v2.0/contributing/data-collection/"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="No default certificate, generating one"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Start TCP Server" entryPointName=websecure
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Start TCP Server" entryPointName=traefik
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Start TCP Server" entryPointName=web
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Starting provider *acme.Provider {\"email\":\"test@test.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"/config/acme.json\",\"keyType\":\"RSA4096\",\"ResolverName\":\"LE\",\"store\":{},\"ChallengeStore\":{}}"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Testing certificate renew..." providerName=LE.acme
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ index .Labels \\\"traefik.subdomain\\\" }}.testserver.nub.local`)\",\"swarmModeRefreshSeconds\":15000000000}"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Configuration received from provider LE.acme: {\"http\":{},\"tls\":{}}" providerName=LE.acme
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="No default certificate, generating one"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Provider connection established with docker 19.03.3 (API 1.40)" providerName=docker
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Filtering disabled container" providerName=docker container=nublab-traefik-1-pm7ndq3uib6sskijxes9l86e2-96cc19ca8588f89954a796c462525e44412775e604b15a42b7864722b3c89013
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:29Z" level=debug msg="Configuration received from provider docker: {\"http\":{},\"tcp\":{}}" providerName=docker
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:30Z" level=debug msg="No default certificate, generating one"
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:58Z" level=debug msg="Provider event received {Status:start ID:629e2fbd27bb58f58314eb7b2747e2fab6807e7b5a4707eb4278ec5aedc0fb2b From:linuxserver/heimdall:latest@sha256:4cf78c5cd1dda5c59a33baabe405c52ad635edeb9eff3d36147faf2b3487ec42 Type:container Action:start Actor:{ID:629e2fbd27bb58f58314eb7b2747e2fab6807e7b5a4707eb4278ec5aedc0fb2b Attributes:map[MAINTAINER:sparkyballs,TheLamer build_version:Linuxserver.io version:- 2.2.2-ls60 Build-date:- 2019-10-04T19:56:57-04:00 com.docker.stack.namespace:nublab com.docker.swarm.node.id:jkr5vac3kkjiugd5o41l90o29 com.docker.swarm.service.id:0whd4x1bh0o0dmxmbxi0tlab9 com.docker.swarm.service.name:nublab_heimdall com.docker.swarm.task: com.docker.swarm.task.id:l75f09blro4uy7ugk3yxhbpto com.docker.swarm.task.name:nublab_heimdall.1.l75f09blro4uy7ugk3yxhbpto image:linuxserver/heimdall:latest@sha256:4cf78c5cd1dda5c59a33baabe405c52ad635edeb9eff3d36147faf2b3487ec42 maintainer:aptalca name:nublab_heimdall.1.l75f09blro4uy7ugk3yxhbpto]} Scope:local Time:1570661698 TimeNano:1570661698659888378}" providerName=docker
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:58Z" level=debug msg="Filtering disabled container" providerName=docker container=nublab-heimdall-1-l75f09blro4uy7ugk3yxhbpto-629e2fbd27bb58f58314eb7b2747e2fab6807e7b5a4707eb4278ec5aedc0fb2b
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:58Z" level=debug msg="Filtering disabled container" container=nublab-traefik-1-pm7ndq3uib6sskijxes9l86e2-96cc19ca8588f89954a796c462525e44412775e604b15a42b7864722b3c89013 providerName=docker
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:58Z" level=debug msg="Configuration received from provider docker: {\"http\":{},\"tcp\":{}}" providerName=docker
nublab_traefik.1.pm7ndq3uib6s@testserver    | time="2019-10-09T22:54:58Z" level=info msg="Skipping same configuration for provider docker" providerName=docker
2

Hello,

You have to enable the swarm mode in Traefik:

[providers.docker]
  endpoint = "unix:///var/run/docker.sock"
  defaultRule = "Host(`{{ index .Labels \"traefik.subdomain\" }}.testserver.nub.local`)"
  exposedByDefault = false
  swarmMode = true
3

Thanks. I feel like an idiot.

What does Traefik do differently in "swarm mode"? Aside from pulling labels from services rather than the containers.

4

So I just enabled swarmMode, but it still doesn't work. It sees the containers now, but it complains:
nublab_traefik.1.ldd5xb6z5z88@testserver | time="2019-10-11T23:18:55Z" level=error msg="port is missing" container=nu blab-heimdall-19re1kxtz4tq549b0xn8h9x6q providerName=docker

Haven't changed anything else. Why do I need to specify the port if it worked before?

5 
---
version: "3.3"

services:
  heimdall:
    image: linuxserver/heimdall
    container_name: heimdall
    volumes:
      - /var/lib/heimdall/config:/config
    restart: unless-stopped
    deploy:
      labels:
          com.centurylinklabs.watchtower.enable: "true"
          traefik.enable: "true"
          traefik.http.routers.heimdall.entrypoints: "websecure"
          traefik.http.routers.heimdall.tls: "true"
          traefik.subdomain: "start"
          traefik.http.services.heimdall.loadbalancer.server.port: 80

https://docs.traefik.io/v2.0/routing/providers/docker/#services

6

Thanks. I'm curious why its a requirement to specify the port but only in swarm mode?

1 Like
7

Hello,

Use docker - compose with docker swarm
Please go through this link, It might help help you: https://stackoverflow.com/questions/39875575/use-docker-compose-with-docker-swarm

Thanks & Regards,
Anita