Does anyone have a sample config of traefik 2.x using consul kv store?

jerryg · April 29, 2021, 10:03pm

I have a fully functioning traefik v1.7.26 using consul as my key value store.

Does anyone have a functioning config for traefik v2.x with using consul as the config store?

I'm failing to migrate, and I think I'm getting a bit wrapped around the axle trying.

cakiwi · April 30, 2021, 2:25pm

I had my first go at a consul + traefik configuration yesterday with [CONSUL][ACL] Using ACL with traefik 2.0 for configuration discovery

Hopefully this can help.

docker-compose traefik+consul

version: "3.8"

services:
  traefik:
    image: "traefik:v2.4"
    command:
    - --entrypoints.web.address=:80
    - --entrypoints.websecure.address=:443
    - --entrypoints.websecure.http.tls=true
    - --providers.docker=true
    - --providers.docker.exposedbydefault=false
    - --providers.consul.endpoints=consul:8500
    - --accesslog
    - --accesslog.format=json
    - --api
    - --log.level=INFO
    - --log.format=json
    labels:
      traefik.enable: "true"
      traefik.http.routers.api.rule: PathPrefix(`/api`) || PathPrefix(`/dashboard`)
      traefik.http.routers.api.service: api@internal
      traefik.http.services.dummy.loadBalancer.server.port: 65535
    ports:
      - published: 80
        target: 80
      - published: 443
        target: 443
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
  whoami:
    image: traefik/whoami
    ports:
    - 8080:80
  consul:
    image: consul
    ports:
    - 8500:8500
    command:
      - agent 
      - -server 
      - -bootstrap 
      - -client=0.0.0.0 
      - -ui

consul KV

curl localhost:8500/v1/kv/traefik/http/services/whoami/loadbalancer/servers/0/url -X PUT -d 'http://172.21.0.2'
curl localhost:8500/v1/kv/traefik/http/routers/kv/rule -X PUT -d 'PathPrefix(`/whoami`)'
curl localhost:8500/v1/kv/traefik/http/routers/kv/service -X PUT -d 'whoami'

jerryg · April 30, 2021, 3:14pm

FYI, using the consul cli, the equivalent would be:

consul kv put , like:

consul kv put traefik/http/services/whoami/loadbalancer/servers/0/url "-X PUT -d 'http://172.21.0.2'"
consul kv put traefik/http/routers/kv/rule "-X PUT -d 'PathPrefix(`/whoami`)'"
consul kv put traefik/http/routers/kv/service "-X PUT -d 'whoami'"

Note that since consul cli is expecting text and two args, you will need to be careful with text qouting

jerryg · April 30, 2021, 8:10pm

What I'm attempting to do is to start traefik by getting traefik's config from consul kv store.
FYI- you have to install and start consul on treafik container, I included a script start-consul.sh.
I also included script to run on consul container to load the keys/vals

But I'm getting this error:
traefik_1 | time="2021-04-30T20:33:21Z" level=error msg="Cannot build the configuration: field not found, node: server" providerName=consul
traefik_1 | time="2021-04-30T20:33:21Z" level=error msg="KV connection error: field not found, node: server, retrying in 279.560325ms" providerName=consul
traefik_1 | time="2021-04-30T20:43:51Z" level=debug msg="Exists: traefik/qmslkjdfmqlskdjfmqlksjazçueznbvbwzlkajzebvkwjdcqmlsfj"
traefik_1 | time="2021-04-30T20:43:51Z" level=debug msg="List: traefik"
traefik_1 | time="2021-04-30T20:43:51Z" level=error msg="Cannot build the configuration: field not found, node: server" providerName=consul
traefik_1 | time="2021-04-30T20:43:51Z" level=debug msg="WatchTree: traefik"
traefik_1 | time="2021-04-30T20:43:51Z" level=debug msg="List: traefik"

docker-compose.yml

version: "3.8"

services:
traefik:
image: "traefik:v2.4"
command:
#- --providers.consulcatalog=true
#- --providers.consulcatalog.prefix=traefik
- --providers.consul=true
- --providers.consul.endpoints=http://localhost:8500
- --accesslog
- --accesslog.format=json
labels:
traefik.enable: "true"
ports:
- published: 82
target: 80
- published: 442
target: 443
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./mydata:/mydata"

whoami:
image: traefik/whoami
ports:
- 8882:80
volumes:
- "./mydata:/mydata"

consul:
image: consul
ports:
- 8502:8500
command:
- agent
- -server
- -bootstrap
- -client=0.0.0.0
- -ui
volumes:
- "./mydata:/mydata"

start-consul.sh

#!/bin/sh

if [ "$#" -eq 1 ]; then
consul_server=$1
else
consul_server=$(grep server /mydata/consul-members.txt | sed s/^[1]\ \ // | sed s/:8301.//)
fi
echo "$0 consul_server: $consul_server"

install consul if not already installed

rc=$(which consul; echo $?)
if [ $rc != "0" ]; then
echo "$0 installing consul"
apk update
apk add --no-cache ca-certificates wget
#update-ca-certificates
export CONSUL_VERSION=1.9.5
wget --no-check-certificate --tries=7 -O /tmp/consul.zip "https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip"
unzip /tmp/consul -d /usr/local/bin
mkdir /consul
mkdir /consul/data
mkdir /consul/config
else
echo "$0 consul found. skipping consul install"
fi

start consul

echo "$0 starting consul agent, attempting to join $consul_server"
/usr/local/bin/consul agent --data-dir /consul/data --config-dir /consul/config -rejoin -retry-join=$consul_server >> /var/log/consul.log 2>&1 &

echo "$0 done"

consul-kv.sh

#!/bin/sh

run this from the console from any node with consul connected to consul server

consul kv put traefik/rootkey traefik
consul kv put traefik/enable true
consul kv put traefik/http/routers/api/rule "PathPrefix('/api') || PathPrefix('/dashboard')"
consul kv put traefik/http/routers/api/service "api@internal"
consul kv put traefik/http/services/dummy/loadBalancer/server/port 65535

consul kv put traefik/entrypoints/web/address ":80"
consul kv put traefik/entrypoints/websecure/address ":443"
consul kv put traefik/entrypoints/websecure/http/tls true
consul kv put traefik/providers/consul true
consul kv put traefik/providers/consul/endpoints "consul:8500"
consul kv put traefik/accesslog/format json
consul kv put traefik/log/level INFO
consul kv put traefik/log/format json

these are from consul +traefik v1.7

consul kv put traefik/consul/endpoint "http://localhost:8500"
consul kv put traefik/consul/prefix traefik
consul kv put traefik/consul/watch true

useful for getting kv keys and vals:

consul kv get -recurse

a-f0-9 ↩︎

cakiwi · May 1, 2021, 2:07pm

You're specifying many invalid keys. Most of which should be in the static configuration.
The reference for KV has the valid keys.

The single quotes ' in a rule have to be backticks `

jerryg · May 13, 2021, 5:40pm

Thank you!

Here are the two things that helped me get unstuck.

Static config. In traefik v1, all of my config was in consul kv store. In traefik v2, I now have no static config in consul or consulCatalog. All of my static config is in a file (I put it in /etc/traefik.toml). That includes letsencrypt acme SSL Cert generation.
Dynamic config. I'm using consulCatalog labels for all of my dynamic config, like Host rules and sticky cookies.

static config - traefik.toml

[global]
checkNewVersion = false
sendAnonymousUsage = false

[serversTransport]
insecureSkipVerify = true

[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.web.http.redirections.entryPoint]
to = "websecure"
scheme = "https"
permanent = false
[entryPoints.websecure]
address = ":443"
[entryPoints.websecure.http.tls]
certResolver = "letsencrypt"
[[entryPoints.websecure.http.tls.domains]]
main = "ACME_MAIN"
sans = ["ACME_SANS"]
[entryPoints.internal-http]
address = ":8081"
[entryPoints.internal-http.http.redirections.entryPoint]
to = "internal-https"
scheme = "https"
permanent = false
[entryPoints.internal-https]
address = ":8443"
[entryPoints.internal-https.http.tls]

[providers]
[providers.consul]
rootKey = "traefikv2"
[providers.consulCatalog]
prefix = "traefik"
exposedByDefault = false

[api]
insecure = true
dashboard = true
debug = true

[certificatesResolvers]
[certificatesResolvers.letsencrypt]
[certificatesResolvers.letsencrypt.acme]
#caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
email = "me@mydomain.com"
storage = "/acme.json"
[certificatesResolvers.letsencrypt.acme.httpChallenge]
entryPoint = "web"

[log]
level = "INFO"

[accessLog]
filePath = "/var/log/traefik-access.log"

dynamic config - sample consulCatalog labels for one service

    "traefik.enable=true"
    ,"traefik.http.routers.wordpress.rule=Host(`mydomain.com`) || Host(`www.mydomain.com`)"
    ,"traefik.http.routers.wordpress.entrypoints=web,websecure"
    ,"traefik.http.services.wordpress.loadbalancer.sticky.cookie=true"
    ,"traefik.http.services.wordpress.loadbalancer.sticky.cookie.httponly=true"
    ,"traefik.http.services.wordpress.loadbalancer.sticky.cookie.samesite=none"
    ,"traefik.http.services.wordpress.loadbalancer.sticky.cookie.secure=true"

Topic		Replies	Views
Traefik v2 acme storage consul Traefik v2 consul-catalog , letsencrypt-acme	1	806	October 1, 2019
Traefik 2.2 with Consul : storeconfig command gone? Traefik v2 docker , docker-swarm , letsencrypt-acme , kv	3	3656	March 30, 2020
Use consul to configure your certificates with Traefik v2 and Swarm Traefik v2 consul-catalog	4	4121	April 27, 2020
Skipping empty Configuration for provider consul Traefik v2 kubernetes-crd , kv	3	1173	April 27, 2020
KV Store in v2.2 Traefik v2 kv	4	1032	March 22, 2020