TLS Handshake Error with Let's Encrypt Certificate via DNS Challenge

I am attempting to have Traefik serve as a reverse proxy for services running in Docker containers. I've been following the documentation and have a small docker environment configured via docker compose that successfully serves data via HTTP. Traefik sits behind HAProxy running in TCP mode forwarding packets received from the Internet to Traefik.

However when I tried to add a new router for serving the same content via HTTPS, I receive the following esoteric (to me) error when I run a curl directed to https://my.domain.tld/: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Full curl output:

 curl -v https://my.domain.tld/
 *   Trying <IP Address of Domain>...
 * TCP_NODELAY set
 * Connected to my.domain.tld (<IP Address of Domain>) port 443 (#0)
 * ALPN, offering h2
 * ALPN, offering http/1.1
 * successfully set certificate verify locations:
 *   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
 * TLSv1.3 (OUT), TLS handshake, Client hello (1):
 * error:1408F10B:SSL routines:ssl3_get_record:wrong version number
 * Closing connection 0

When I attempt to browse to the site via Firefox (web browser) I receive an error code of SSL_ERROR_RX_RECORD_TOO_LONG. When googling this error I was unable to find a post that seemed to have my specific issue.

Below is the docker-compose for the setup I am using to configure the applications

version: "3.9"

secrets:
  cloudflare_dns_token:
    file: ./secrets/cf_dns_api_token.txt

networks:
  socket_proxy:
    name: socket_proxy
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.0.0/24
  container_bridge:
    name: container_bridge
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.1.0/24

services:
  socket-proxy:
    image: tecnativa/docker-socket-proxy
    container_name: socket-proxy
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      socket_proxy:
        ipv4_address: 192.168.0.2 # Static IP address
    environment:
      EVENTS: 1
      PING: 1
      VERSION: 1
      CONTAINERS: 1
      NETWORKS: 1
  
  traefik:
    # The official v2 Traefik docker image
    image: traefik:v2.8.1
    container_name: traefik-proxy
    command:
      # Log Level for Traefik 
      - "--log.level=DEBUG" 
      # Enables the web UI 
      - "--api.insecure=true"
      # Traefik enables docker as the provider to look for services
      - "--providers.docker=true"
      # Traefik will use the Docker Socket proxy to communicate with the docker socket
      - "--providers.docker.endpoint=tcp://192.168.0.2:2375"
      # Traefik will not expose services if they aren't labled for export
      - "--providers.docker.exposedByDefault=false"
      # Port where Traefik will listen for web (http) traffic for routing
      - "--entrypoints.web.address=:80"
      # Port where Traefik will listen for web secure (https) traffic for routing
      - "--entrypoints.websecure.address=:443"
      # Trust Proxy Protocol Packets from only the listed IP address
      - "--entryPoints.web.proxyProtocol.trustedIPs=10.0.8.1/32"
      # Trust Proxy Protocol Packets from only the listed IP address
      - "--entryPoints.websecure.proxyProtocol.trustedIPs=10.0.8.1/32"
      # Enable a ACME DNS challenge named "letsencrypt"
      - "--certificatesresolvers.letsencrypt.acme.dnschallenge=true"
      # Tell Traefik which provider to use for DNS Challenge
      - "--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare"
      # Staging environment for let's encrypt for testing
      - "--certificatesresolvers.letsencrypt.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
      # Email to provide to let's encrypt
      - "--certificatesresolvers.letsencrypt.acme.email=${EMAIL}"
      # Tell Traefik to store the certificate on a path under our volume
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    networks:
      # Tells Traefik to connect to both the socket proxy network and the container bridge network where the other containers will be connected
      socket_proxy:
      container_bridge:
    ports:
      # The HTTP port
      - "80:80"
      # The HTTPS port
      - "443:443"
      # The Web UI (enabled by --api.insecure=true)
      - "8080:8080"
    secrets:
      - "cloudflare_dns_token"
    environment:
      # To Be Removed Need Secret Working Properly
      - "CF_DNS_API_TOKEN=${CF_DNS_TOKEN}"
      #- "CF_DNS_API_TOKEN=/run/secrets/cloudflare_dns_token"
    volumes:
      # Create a letsencrypt dir within the folder where the docker-compose file is
      - "./letsencrypt:/letsencrypt"
  
  whoami:
    image: traefik/whoami
    container_name: whoami-server
    networks:
      container_bridge:
    labels:
      # Tells Traefik to proxy to the service (container)
      - "traefik.enable=true"
      #####################################################################
      #
      # Labels for HTTPS Proxying
      #
      #####################################################################
      # Explicitly stating 'whoami-secure' route is HTTPS
      - "traefik.http.routers.whoami-secure.tls=true"
      # Rule for determing when to route requests to this service for the secure http router
      - "traefik.http.routers.whoami-secure.rule=Host(`whoami.${FQDN}`)"
      # Entry point for requests to this service for the secure http router
      - "traefik.http.routers.whoami-secure.entrypoints=websecure"
      # Uses the Host rule to define which certificate to issue
      - "traefik.http.routers.whoami-secure.tls.certresolver=letsencrypt"
      #####################################################################
      #
      # Labels for HTTP Proxying
      #
      #####################################################################
      # Rule for determing when to route requests to this service for the unsecure http router
      - "traefik.http.routers.whoami.rule=Host(`whoami.${FQDN}`)"
      # Entry point for requests to this service for the unsecure http router
      - "traefik.http.routers.whoami.entrypoints=web"

My expectation is that Traefik would gracefully handle the request via HTTPS and manage the TLS handshake without issue. I can confirm that Traefik is able to successfully generate a certificate via Let's Encrypt DNS Challenge for Cloudflare. I am using the Let's Encrypt staging environment at the moment so I did expect an error about the certificate being served as invalid, but it seems that TLS handshake errors out before a determination of validity.

Any thoughts or suggestions on this error are appreciated!

For future folks who stumble upon this issue. I solved the issue by redoing my HAProxy config. After hitting Traefik from the box that was running HAProxy I noticed Traefik was properly serving certificates. This allowed me to correctly identify that HAProxy was the cause of the error.

I changed my HAProxy config to what's found below and my issues disappeared

listen https-in
        bind *:443
        mode tcp
        server reverse-proxy-vm <internal IP of backend>:443 send-proxy

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.