Newbie alert.
I have managed , with lots of help from Techno Tim (Youtube) and Jim's Garage (Youtube) to setup traefik for docker containers for a single proxmox VM. I read that to enable reverse proxy to other nodes, I should setup a docker swarm , which I also setup for 5 nodes
I do not know where to start on editing my docker-compose, and traefik.yml even after reading quite a big as its quite confusing.
Hopefully, some kind forumers can help me.
docker-compose.yaml
services:
traefik:
image: traefik:latest
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- proxy
ports:
- 80:80
- 443:443
# - 443:443/tcp # Uncomment if you want HTTP3
# - 443:443/udp # Uncomment if you want HTTP3
environment:
CF_DNS_API_TOKEN_FILE: /run/secrets/cf_api_token # note using _FILE for docker secrets
# CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN} # if using .env
TRAEFIK_DASHBOARD_CREDENTIALS: ${TRAEFIK_DASHBOARD_CREDENTIALS}
secrets:
- cf_api_token
env_file: .env # use .env
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /opt/docker/traefik/data/traefik.yml:/traefik.yml:ro
- /opt/docker/traefik/data/acme.json:/acme.json
- /opt/docker/traefik/logs:/var/log/traefik
- /opt/docker/traefik/data/config.yml:/config.yml:ro
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.routers.traefik.rule=Host(traefik-dashboard.mydomain
)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_DASHBOARD_CREDENTIALS}"
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=https"
- "traefik.http.routers.traefik-secure.rule=Host(traefik-dashboard.mydomain
)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
- "traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain"
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain"
- "traefik.http.routers.traefik-secure.service=api@internal"
secrets:
cf_api_token:
file: ./cf_api_token.txt
networks:
proxy:
external: true
traefik.yml
api:
dashboard: true
debug: true
entryPoints:
http:
address: ":80"
http:
# middlewares: # uncomment if using CrowdSec - see my video
# - crowdsec-bouncer@file
redirections:
entrypoint:
to: https
scheme: https
https:
address: ":443"
# http:
# middlewares: # uncomment if using CrowdSec - see my video
# - crowdsec-bouncer@file
tcp:
address: ":10000"
apis:
address: ":33073"
serversTransport:
insecureSkipVerify: true
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
file:
filename: /config.yml # example provided gives A+ rating SSL Server Test (Powered by Qualys SSL Labs)
certificatesResolvers:
cloudflare:
acme:
caServer: https://acme-v02.api.letsencrypt.org/directory # production (default)
#caServer: https://acme-staging-v02.api.letsencrypt.org/directory # staging (testing)
email: myemail@email.com # Cloudflare email (or other provider)
storage: acme.json
dnsChallenge:
provider: cloudflare # change as required
disablePropagationCheck: true # uncomment this if you have issues pulling certificates through cloudflare, By setting this flag to true disables the need to wait for the propagation of the TXT record to all authoritative name servers.
delayBeforeCheck: 60s # uncomment along with disablePropagationCheck if needed to ensure the TXT record is ready before verification is attempted
resolvers:
- "1.1.1.1:53"
- "1.0.0.1:53"
log:
level: "ERROR"
filePath: "/var/log/traefik/traefik.log"
accessLog:
filePath: "/var/log/traefik/access.log"
Use 3 backticks before and after your code/config to make it more readable and preserve spacing, which is important in yaml.
Check simple Traefik Swarm example.
Main changes:
- Switch from
providers.docker
to providers.swarm
- Place labels within
deploy
section
- Add
loadbalancer.server.port
- Use
docker stack deploy
- When running multiple Traefik instances, you can’t use LetsEncrypt
httpChallenge
or tlsChallenge
Note that Docker Swarm will not create bind mount folders on host, like Docker does when they don’t exist. Also note that volumes are only on the node, they are not replicated to other nodes.
Thank you!
Please assume I am a total idiot.
Would these amendments be correct?
version: "3.8"
services:
traefik:
image: traefik:latest
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- proxy
ports:
- 80:80
- 443:443
# - 443:443/tcp # Uncomment if you want HTTP3
# - 443:443/udp # Uncomment if you want HTTP3
environment:
CF_DNS_API_TOKEN_FILE: /run/secrets/cf_api_token # note using _FILE for docker secrets
# CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN} # if using .env
TRAEFIK_DASHBOARD_CREDENTIALS: ${TRAEFIK_DASHBOARD_CREDENTIALS}
secrets:
- cf_api_token
env_file: .env # use .env
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./data/traefik.yml:/traefik.yml:ro
- ./data/acme.json:/acme.json
- ./data/logs:/var/log/traefik
- ./data/config.yml:/config.yml:ro
command:
- --providers.swarm.exposedByDefault=false
- --providers.swarm.network=proxy
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entrypoints.web.http.redirections.entrypoint.scheme=https
- --entrypoints.web.address=:443
- --entrypoints.websecure.asDefault=true
- --entrypoints.websecure.http.tls.certresolver=myresolver
- --entrypoints.websecure.http.tls.domains[0].main=domain.com
- --entrypoints.websecure.http.tls.domains[0].sans=*.domain.com
- --certificatesresolvers.myresolver.acme.email=email@email.com
- --certificatesresolvers.myresolver.acme.storage=/data/acme.json
- --certificatesresolvers.myresolver.acme.dnschallenge.provider=cloudflare
deploy:
mode: global
placement:
constraints:
- node.role==manager
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.services.traefik-secure.loadbalancer.server.port=1337
- "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.domain.com`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_DASHBOARD_CREDENTIALS}"
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=https"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.domain.com`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
- "traefik.http.routers.traefik-secure.tls.domains[0].main=domain.com"
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.domain.com"
- "traefik.http.routers.traefik-secure.service=api@internal"
secrets:
cf_api_token:
file: ./cf_api_token.txt
networks:
proxy:
external: true
Also, do I need any changes in the traefik.yml file?
You can only have one Traefik static config, either traefik.yml
or command:
(doc), decide for one.
You don’t need the https redirect labels, if redirect is already on web entrypoint
. Also TLS declaration is enough on websecure entrypoint
.
And I don’t get it why every newbie comes along with this line, it’s absolutely useless, as it’s set by Traefik automatically:
customrequestheaders.X-Forwarded-Proto=https
Thank you.
So if I wish to continue using traefik.yml, how do I adapt the entire command: section needed in my current traefik.yml file?
api:
dashboard: true
debug: true
entryPoints:
http:
address: ":80"
http:
redirections:
entryPoint:
to: https
scheme: https
https:
address: ":443"
serversTransport:
insecureSkipVerify: true
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
file:
filename: /config.yml
certificatesResolvers:
cloudflare:
acme:
email: email@email.com
storage: acme.json
caServer: https://acme-v02.api.letsencrypt.org/directory # prod (default)
# caServer: https://acme-staging-v02.api.letsencrypt.org/directory # staging
dnsChallenge:
provider: cloudflare
disablePropagationCheck: true # uncomment this if you have issues pulling certificates through cloudflare, By setting this flag to true disables the need to wait for the propagation of the TXT record to all authoritative name servers.
delayBeforeCheck: 60s # uncomment along with disablePropagationCheck if needed to ensure the TXT record is ready before verification is attempted
resolvers:
- "1.1.1.1:53"
- "1.0.0.1:53"
log:
level: "ERROR"
filePath: "/var/log/traefik/traefik.log"
Also, is it also possible to have just one instance of traefik running for all my hosts?
Sorry, i will not check your config line by line. You can take additional lines from command and work them into traefik.yml. I created the repo to have a best practice starting point.
You can have a single Traefik instance for Swarm. You can pin it to a host with placement constraints, or set replica: 1
. Simple example for single manager.
When the port/container is not explicitly in network mode "host", then Swarm will create an ingress network and the instance can be reached from any node on the published port.
For a stable Swarm, you should have 3 manager nodes, which will also act as workers.
Thanks so much!
Noted on the 3 node swarm requirements
Would I also need shared storage for the Swarm to be stable? eg an NFS share from truenas?
Swarm just works. But it has no shared storage.
DO you mean to remove from here?
ports:
- target: 80
published: 80
protocol: tcp
#mode: host
- target: 443
published: 443
protocol: tcp
#mode: host
with mode: host
commented out, the ingress network should be created, and your service should be available on all nodes on the ports.
1 Like
I am unable to deploy the stack , with these errors:
Since --detach=false was not specified, tasks will be created in the background.
In a future release, --detach=false will become the default.
network "proxy" is declared as external, but it is not in the right scope: "loca l" instead of "swarm"
My docker-compose.yml :
services:
traefik:
image: traefik:v3.3
hostname: '{{.Node.Hostname}}'
networks:
- proxy
ports:
- target: 80
published: 80
protocol: tcp
#mode: host
- target: 443
published: 443
protocol: tcp
#mode: host
environment:
CF_DNS_API_TOKEN_FILE: /run/secrets/cf_api_token # note using _FILE for docker secrets
# CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN} # if using .env
TRAEFIK_DASHBOARD_CREDENTIALS: ${TRAEFIK_DASHBOARD_CREDENTIALS}
secrets:
- cf_api_token
env_file: .env # use .env
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
# - /opt/docker/traefik/data/traefik.yml:/traefik.yml:ro
- /opt/docker/traefik/data/acme.json:/acme.json
- /opt/docker/traefik/logs:/var/log/traefik
# - /opt/docker/traefik/data/config.yml:/config.yml:ro
command:
- --api.dashboard=true
- --api.debug=true
- --log.level=DEBUG
- --log.filepath=/var/log/traefik.log
- --accesslog=true
- --accesslog.filepath=/var/log/traefik-access.log
- --providers.swarm.exposedByDefault=false
- --providers.swarm.network=proxy
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entryPoints.web.http.redirections.entrypoint.scheme=https
- --entrypoints.websecure.address=:443
- --entrypoints.websecure.asDefault=true
- --entrypoints.websecure.http.tls.certresolver=cloudflare
# optionally create wildcard cert, without LE will create TLS certs for all `Host()`s
- --entrypoints.websecure.http.tls.domains[0].main=mydomain.com
- --entrypoints.websecure.http.tls.domains[0].sans=*.mydomain.com
- --certificatesresolvers.cloudflare.acme.email=myemail@email.com
- --certificatesresolvers.cloudflare.acme.storage=acme.json
- --certificatesresolvers.cloudflare.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory # production
- --certificatesresolvers.cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53
- --certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare
- --serversTransport.insecureSkipVerify=true
deploy:
mode: global
replicas: 1
placement:
constraints:
- node.role==manager
restart_policy:
condition: unless-stopped
delay: 2s
max_attempts: 3
window: 60s
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.mydomain.com`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_DASHBOARD_CREDENTIALS}"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.mydomain.com`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
- "traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain.com"
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain.com"
- "traefik.http.routers.traefik-secure.service=api@internal"
- "traefik.http.services.traefik-secure.loadbalancer.server.port=1337"
whoami:
image: traefik/whoami:v1.10
hostname: '{{.Node.Hostname}}'
networks:
- proxy
deploy:
mode: global
labels:
- traefik.enable=true
- traefik.http.routers.whoami.rule=Host(`whoami.mydomain.com`)
- traefik.http.services.whoami.loadbalancer.server.port=80
secrets:
cf_api_token:
file: ./cf_api_token.txt
networks:
proxy:
name: proxy
driver: overlay
attachable: true
external: true
I must be missing something?
This seems wrong. Either you create the network within the compose project (driver
, attachable
), or you use an already existing one (external
).
And you probably have an old local "proxy" named Docker network on host, you need to remove that (docker network ls
, docker network rm proxy
).
Note that you don’t need any TLS settings on router
/labels, if they are already set globally on entrypoint
.
Thank you!
so i just amend this section like this:
networks:
proxy:
name: proxy
driver: overlay
attachable: true
Will remove the old local "proxy" network
What do you mean by TLS settings on router.labels? you mean the 3 lines with .tls in them?
Yes. If TLS is set up globally on entrypoint
in static config, you don't need to repeat in every dynamic config on every router
. Check simple Traefik dnsChallenge example.
1 Like
have this error after * Run docker stack deploy -c docker-compose.yml myProxy
Since --detach=false was not specified, tasks will be created in the background.
In a future release, --detach=false will become the default.
Creating network proxy
Creating secret myProxy_cf_api_token
service traefik: replicas can only be used with replicated or replicated-job mode
Share your current compose file.
services:
traefik:
image: traefik:v3.3
hostname: '{{.Node.Hostname}}'
networks:
- proxy
ports:
- target: 80
published: 80
protocol: tcp
#mode: host
- target: 443
published: 443
protocol: tcp
#mode: host
environment:
CF_DNS_API_TOKEN_FILE: /run/secrets/cf_api_token # note using _FILE for docker secrets
# CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN} # if using .env
TRAEFIK_DASHBOARD_CREDENTIALS: ${TRAEFIK_DASHBOARD_CREDENTIALS}
secrets:
- cf_api_token
env_file: .env # use .env
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
# - /opt/docker/traefik/data/traefik.yml:/traefik.yml:ro
- /opt/docker/traefik/data/acme.json:/acme.json
- /opt/docker/traefik/logs:/var/log/traefik
# - /opt/docker/traefik/data/config.yml:/config.yml:ro
command:
- --api.dashboard=true
- --api.debug=true
- --log.level=DEBUG
- --log.filepath=/var/log/traefik.log
- --accesslog=true
- --accesslog.filepath=/var/log/traefik-access.log
- --providers.swarm.exposedByDefault=false
- --providers.swarm.network=proxy
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entryPoints.web.http.redirections.entrypoint.scheme=https
- --entrypoints.websecure.address=:443
- --entrypoints.websecure.asDefault=true
- --entrypoints.websecure.http.tls.certresolver=cloudflare
# optionally create wildcard cert, without LE will create TLS certs for all `Host()`s
- --entrypoints.websecure.http.tls.domains[0].main=mydomain.com
- --entrypoints.websecure.http.tls.domains[0].sans=*.mydomain.com
- --certificatesresolvers.cloudflare.acme.email=myemail@email.com
- --certificatesresolvers.cloudflare.acme.storage=acme.json
- --certificatesresolvers.cloudflare.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory # production
- --certificatesresolvers.cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53
- --certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare
- --serversTransport.insecureSkipVerify=true
deploy:
mode: global
replicas: 1
placement:
constraints:
- node.role==manager
restart_policy:
condition: unless-stopped
delay: 2s
max_attempts: 3
window: 60s
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.mydomain.com`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_DASHBOARD_CREDENTIALS}"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.mydomain.com`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
- "traefik.http.routers.traefik-secure.tls.domains[0].main=mydomain.com"
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.mydomain.com"
- "traefik.http.routers.traefik-secure.service=api@internal"
- "traefik.http.services.traefik-secure.loadbalancer.server.port=1337"
whoami:
image: traefik/whoami:v1.10
hostname: '{{.Node.Hostname}}'
networks:
- proxy
deploy:
mode: global
labels:
- traefik.enable=true
- traefik.http.routers.whoami.rule=Host(`whoami.mydomain.com`)
- traefik.http.services.whoami.loadbalancer.server.port=80
secrets:
cf_api_token:
file: ./cf_api_token.txt
networks:
proxy:
name: proxy
driver: overlay
attachable: true
type or paste code here
This does not work. Either mode: global
, so one container on every node, or replicas:
to determine the number of containers in the whole Swarm.
thank you.
Is there a sequence I should follow to prepare for this new configuration, as my current swarm has a single traefik instance on one of the nodes,and the last time disabled it and tried to do a new traefik (with swarm mode modifications), that node could not rejoin the swarm again?
You can run regular containers even on Swarm nodes. So you can keep the single Traefik instance running, establish a Swarm, stop the container, start a Docker service.
If it doesn’t work the way you want it, just stop the service and start the single container again. No influence on Swarm.