Haproxy in front of Traefik and Docker with Letsencrypt

Hi all
I setup docker and traefik with letsencrypt on my vps and everything worked fine. Certificates were created for my Traefik dashboard, whoami test app and a subdomain of my main domain.
But I wanted a dedicated load balancer in from of this setup, so I obtained another vps and installed Haproxy. I configured haproxy as per the instructions. I am now able to access the dashboard, whoami and my subdomain. I also see that the acme.json is populated but my connections remain unsecured. Can anybody show me what I am doing wrong? Here are my haproxy.cfg and docker compose file for traefik. Thank you in advance.
(haproxy.cfg)

defaults
        log global
        mode tcp
        option tcplog
frontend loadbalancers
   bind *:80
   bind *:443
   default_backend mngrsHTTPS
backend mngrsHTTPS
    balance roundrobin
    mode tcp
    option ssl-hello-chk
    server mngr1 <ip-address>:443 check
    server mngr2 <ip-address>:443 check

(docker compose for traefik)

command:
      - "--api.dashboard=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.https.address=:443"
      # TLS certificates resolvers
      - "--certificatesresolvers.myresolver.acme.tlschallenge=true"
      - "--certificatesresolvers.myresolver.acme.email=me@mail.com"
      - "--certificatesresolvers.myresolver.acme.storage=~/web/letsencrypt/acme.json"
      - "--entrypoints.web.http.redirections.entryPoint.to=https"
      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
      # Uncomment the below two lines while testing/staging. Comment out for full letsencrypt tls
      - "--log.level=DEBUG"
      - "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.tls=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.mysite.com`)"
      - "traefik.http.routers.traefik.entrypoints=https"
      - "traefik.http.routers.traefik.tls.certresolver=myresolver"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.middlewares=traefik-basic-auth"
      - "traefik.http.middlewares.traefik-basic-auth.basicauth.users=<username>:<password>"
    expose:
      # traefik dashboard port
      - "8383"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "~/web/letsencrypt:/letsencrypt"
    networks:
      - "vpsNet"

Which one?

  1. external to Haproxy
  2. Haproxy to Traefik
  3. Traefik to target service

Thanx for your reply.
When I visit the Traefik dashboard [https://dashboard.mysite.com], it's unsecured. Browser warning.
When I visit https://whoami.mysite.com, it's unsecured.
Like I said, before I added haproxy load balancer in front of docker all worked ok.
Thanx again

I am not familiar with the haproxy configuration. What does it do? Just plain forward the TCP/IP connection? To me it seems you forward port 80 and 443 to Traefik 443. We have an LB in front of Traefik, we forward 80->80 and 443->443. (But we use custom paid wildcard certs)

Your main problem is probably distributed Traefik. You can not (easily) use LetsEncrypt with multiple Traefik instances, because one will trigger the verification, but the verification code is not available within the other instance, which might receive the verification request.

Traefik EE support distributed LE, requires a subscription. There is an experimental open-source solution with an additional single certbot instance, see here.

The other option would be to let the first proxy (haproxy) handle TLS termination. You could still forward with https with a custom cert to the Traefik instances, haproxy just needs to trust them.

Thank you very much for your informative reply. I will look into your suggestions...
But I was wondering. Is it possible to have a traefik in front of traefik, instead of haproxy? Like a stand alone traefik which acts as load balancer to docker in the backend?

Yes, that should work.

You could just use Docker Swarm to connect your nodes to a „swarm“ and have a single Traefik distribute request to all nodes.