Docker swarm with Traefik 2 as proxy, adding new node with already existing webserver

Traefik Traefik v2 (latest)
1

I run a small docker swarm cluster wit 4 nodes (1 manager/leader, 3 workers). The manager node has Traefik2 installed. I've wanted to add a new node, a server already running an apache server with a couple of websites secured with let's encrypt certificates. Immediately after adding the node with docker swarm join, traefik takes over and replaces all the certificates with his own self signed certificate. I know this is expected behavior, but can I somehow prevent it and run an existing webserver with traefik ?

Here is my traefik.yml

version: '3.3'

services:

  traefik:
    image: traefik:latest
    ports:
      - 80:80
      - 443:443
    deploy:
      placement:
        constraints:
          - node.labels.traefik-public.traefik-public-certificates == true
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.middlewares.admin-auth.basicauth.users=user:pass
        - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
        - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
        - traefik.http.routers.traefik-public-http.rule=Host(`traefik.example.comm`)
        - traefik.http.routers.traefik-public-http.entrypoints=http
        - traefik.http.routers.traefik-public-http.middlewares=https-redirect
        - traefik.http.routers.traefik-public-https.rule=Host(`traefik.example.com`)
        - traefik.http.routers.traefik-public-https.entrypoints=https
        - traefik.http.routers.traefik-public-https.tls=true
        - traefik.http.routers.traefik-public-https.service=api@internal
        - traefik.http.routers.traefik-public-https.tls.certresolver=le
        - traefik.http.routers.traefik-public-https.middlewares=admin-auth
        - traefik.http.services.traefik-public.loadbalancer.server.port=8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-public-certificates:/certificates
    command:
      - --providers.docker
      - --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
      - --providers.docker.exposedbydefault=false
      - --providers.docker.swarmmode
      - --entrypoints.http.address=:80
      - --entrypoints.https.address=:443
      - --certificatesresolvers.le.acme.email=mail@example.com
      - --certificatesresolvers.le.acme.storage=/certificates/acme.json
      - --certificatesresolvers.le.acme.tlschallenge=true
      - --accesslog
      - --log
      - --api
    networks:
      - traefik-public

volumes:
  traefik-public-certificates:

networks:
  traefik-public:
    external: true
2

Hello.
The answer is yes and no.
Because you have docker swarm. Traefik have to be on docker master(s) .
part of my *compose.yml

    networks:
#      - prometheus
      - traefiknet
#      - dmznet
    logging:
      options:
        max-size: '12m'
        max-file: '5'

    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: any

traefik assigned to manager(s)
constraints:
- node.role == manager

So you can assign your containers(services) to workers
constraints:
- node.role == worker

BTW. only manager take care about incoming requests, workers forward requests to managers.

3

HI,
Traefik is on the manager/leader node.

node.labels.traefik-public.traefik-public-certificates == true

this tag was made on the node so it always deploys on the same manager node.

The web server is on a worker node. According to your explanation the flow is like that ?
node1 - master/leader with traefik, IP 1.1.1.1
node2 - worker node , IP 2.2.2.2

DNS A record for www.example.com points to 2.2.2.2. There is an apache2 webserver outside of docker. On the worker node a request on port 80 or 443 comes in. It doesn't reach the apache webserver, it's forwarded to the manager, traefik hasn't any rules about that domain and forwards the request back to node2. It reaches the external webserver on node2, but with a self signed Traefik certificate. Is there a workaround, some kind of whitelist, blacklist ? Or did I misunderstood your answer ?

4

Lets do next: we dont ha

ve traefik
apache2 web server run on worker node with ip 2.2.2.2 and publish ports 80:80;443:443 (certificate is correct)
DNS A record points to 1.1.1.1 master node

It it working correct ?

BTW

use volumes in docker swarm is not good idea.
can be better to use mounted nfs share for example mounted to /external folder

volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /external/services/traefik/traefik-public-certificates:/certificates

One more traefik can provide let's encrypt certificates for your servers.

Also i recommend to have 3 masters and 1+ worker.

Add manager nodes for fault tolerance

Swarm Size Majority Fault Tolerance
1 1 0
2 2 0
3 2 1
5

Here is my docker-compose.yml

version: "3.4"
services:
  proxy:
    image: traefik
    command:
      - "--api=true"
      - "--api.debug=true"
      - "--api.dashboard=true"
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.swarmmode=true"
      - "--providers.docker.network=traefiknet"
      - "--providers.docker.watch=true"
      - "--providers.file.debugloggeneratedtemplate=true"
      - "--providers.file.watch=true"
      - "--providers.file.directory=/etc/traefik/conf"
##?
#      - "--providers.docker.exposedbydefault=true"
##?
##      - "--providers.file.filename=/etc/traefik/traefik.toml"
      - "--accesslog=false"
      - "--accesslog.filepath=/var/log/traefik_access.log"
      - "--metrics.prometheus=true"
      - "--metrics.prometheus.addentrypointslabels=true"
      - "--metrics.prometheus.addserviceslabels=true"
      - "--metrics.prometheus.buckets=0.100000, 0.300000, 1.200000, 5.000000"
#######################################################
      - "--ping=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
##
      - "--certificatesResolvers.le.acme.email=tracingsss@gmail.com"
      - "--certificatesResolvers.le.acme.storage=/etc/traefik/acme/acme.json"
      - "--certificatesResolvers.le.acme.tlsChallenge=true"
      - "--certificatesResolvers.le.acme.httpChallenge=true"
      - "--certificatesResolvers.le.acme.httpChallenge.entryPoint=web"

##
#######################################################
      - "--log=true"
      - "--log.filepath=/var/log/traefik.log"
#      - "--log.level=DEBUG"
      - "--log.level=ERROR"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /external/efs/services/traefik/acme/acme.json:/etc/traefik/acme/acme.json
#      - /external/efs/services/traefik/traefik.toml:/etc/traefik/traefik.toml
      - /external/efs/services/traefik/log:/var/log/
      - /external/efs/services/traefik/conf:/etc/traefik/conf
    labels:
      - traefik.enable=true
#      - "traefik.http.routers.api.entryPoints=traefik"
#      - "traefik.http.routers.api.rule=PathPrefix(`/api`) || PathPrefix(`/dashboard`)"
#      - "traefik.http.routers.api.service=api@internal"
#      - "traefik.http.routers.api.middlewares=api-auth"
#      - "traefik.http.middlewares.api-auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6EWTrSuBkTrqE8wj/" # test:test
##################3
#      - traefik.docker.lbswarm=true
#      - traefik.docker.network=traefiknet
#      - traefik.http.routers.traefik.rule=Host(`proxy.example.me`)
#      - traefik.http.routers.traefik.tls=true
#      - traefik.http.routers.traefik.entrypoints=web
#      - traefik.http.routers.traefik.tls.certresolver=le
#      - traefik.http.routers.traefik.service=traefik
#      - traefik.http.services.traefik.loadbalancer.server.port=8080
#      - traefik.http.routers.traefik.middlewares=traefik-auth
#      - traefik.http.middlewares.traefik-auth.basicauth.users=admin:$$apr1$$8EVjn/nj$$GiLUZqcBuETFeDD3SuB6x0
# Redirect all HTTP to HTTPS permanently
      - traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)
      - traefik.http.routers.http_catchall.entrypoints=web
      - traefik.http.routers.http_catchall.middlewares=https_redirect
      - traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https_redirect.redirectscheme.permanent=true

    networks:
#      - prometheus
      - traefiknet
#      - dmznet
    logging:
      options:
        max-size: '12m'
        max-file: '5'

    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
    deploy:
      mode: global
      placement:
        constraints:
          - node.role == manager
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: any

networks:
#  prometheus:
#    driver: overlay
#    external: true
#      name: dockerprometheus_back-tier
#  dmznet:
#    driver: overlay
#    external: true
  traefiknet:
    driver: overlay
    attachable: true
    external: true

also some records from script:

echo "Create the overlay network that will be used for our stack (traefiknet):"
docker network create --driver overlay --attachable traefiknet

ghost-compose.yml

version: '3.4'

services:
  blog:
    image: ghost
    deploy:
      mode: replicated
      replicas: 2
      placement:
        constraints:
          - node.role == worker
      labels:
        - traefik.enable=true
        - traefik.docker.lbswarm=true
        - traefik.docker.network=traefiknet
        - traefik.http.routers.blog.rule=Host(`blog.example.me`)
        - traefik.http.routers.blog.tls=true
        - traefik.http.routers.blog.tls.certresolver=le
        - traefik.http.routers.blog.entrypoints=web,websecure
        - traefik.http.routers.blog.service=blog
        - traefik.http.services.blog.loadbalancer.server.port=2368
        - traefik.http.services.blog.loadbalancer.server.scheme=http
        - traefik.http.services.blog.loadbalancer.passhostheader=true
        - traefik.http.services.blog.loadbalancer.sticky=true
    ports:
      - 1192:2368
    networks:
      - traefiknet


networks:
  traefiknet:
    external: true
7

Nice, thanks for the doc. WIll look into it.
So, I've added node2 to the swarm and i got the certificate error, then I removed the traefik stack and the hp's outside docker work fine, of course dockerized applications don;t work anymore. DNS points to 2.2.2.2 though.

8

To remove the possibility of misunderstanding. Here is my complete environment (I will change that according to your recommedation though)

node1 : master node, leader, 1.1.1.1
dockerized traefik, grafana, prometheus and portainer with agents.
One vpn server that's not dockerized
node2: worker node, ip 2.2.2.2
no dockerized apps at the moment since the problem with the certificate shows ass soon as I add it to the swarm
existing server with db, apache webserver dns, webserver listening on 80,443 tcp
node3: worker node, ip 3.3.3.3
dockerized gitlab
existing mailserver and openvpn server, since they don't use ports 80 and 443, everything works fine with traefik

all domains/subdomains like grafana, git portrainer work fine since they are dockerized and they run thru traefik and the stacks have according labels

domain A record for www.new-example.com points to ip 2.2.2.2 (since there is the existing webserver). The webserver handles https and certificates. When activating traefik, apparently everything on port 80,443, that comes in on node2 is forwarded to the manager node1 with traefik on it. Traefik doesn't find a configuration for www.new-example.com since there is nothing there because it's outside docker and it forwards the request back to node2 but with traefik's own selfsigned certificate. The homepage works, but I get a warning about the site is not secure and I have to add an exception if I want to see it. With HTST I can't even do that because it doesn't allow adding exceptions.

I happen to have a couple of servers with existing services on it. I've wanted to try out docker in swarm mode with a convenient reverse proxy like Traefik and I used those servers to play around with it. I know it's not a common configuration to mix dockerized apps and existing services (or at least I don;t think it is common). SO if it doesn't work that way I'll use only clean servers for docker and traefik in the future.

9

The solution to get traefik working with existing webservers is to make Traefik listen directly, not through Docker Swarm mode.
As @SkazochnikZlodey already pointed it out in his first reply and I made a mess in the thread after that :slight_smile: , you need to publish the ports in host mode.

    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host

Thanks for the help @SkazochnikZlodey

10

Traefik also use config for external servers.

  • "--providers.file.watch=true"
    - "--providers.file.directory=/etc/traefik/conf"

  • /external/efs/services/traefik/conf:/etc/traefik/conf

and protect site with ssl

kibana.yml

http:
  routers:
    kibana_development-http:
      rule: "Host(`kibana.example.me`)"
      entryPoints:
        - web
      service: kibana_development_me
      middlewares:
        - kibana_development-redirect
    kibana_development-https:
      rule: "Host(`kibana.example.me`)"
      entryPoints:
        - websecure
      service: kibana_development_me
      middlewares:
        - kibana-auth
      tls:
        certResolver: le
  services:
    kibana_development_me:
      loadBalancer:
        servers:
          - url: "http://10.0.10.233:5601"
  middlewares:
    kibana_development-redirect:
      redirectScheme:
        scheme: https
    kibana-auth:
      basicAuth:
        removeHeader: true
        users:
          - "admin:{SHA}ig1LQaV1NBIjFkSeA/3RvUAx31Q="
1 Like
11

This is not a general rule, this is only true in the specific circumstance where a node in the swarm is also running a singleton container that is publishing a port that you are publishing with Traefik.

1 Like
12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.