Why does TCP router try to perform SSL handshake even if TLS is set to false?

Traefik Traefik v2
,
1

What did you do?

We are using Traefik v2.9.10. We would like to deploy an application which would respond to TCP requests on entrypoint 8093 (backend-thrift-tcp). It must work without TLS. Our Traefik is configured like this:

docker-compose-Traefik.yml:

version: "3.7"

services:
  traefik:
    image: "traefik:v2.9.10"
    networks:
      - traefik-net
    ports:
      # Traefik
      - target: 9000
        published: 9000
        protocol: tcp
        mode: host
        
      # backend
      - target: 8082
        published: 8082
        protocol: tcp
        mode: host
        
      # backend-thrift
      - target: 8092
        published: 8092
        protocol: tcp
        mode: host
        
      # backend-thrift-tcp
      - target: 8093
        published: 8093
        protocol: tcp
        mode: host

    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./TraefikConfig/:/etc/traefik/"
      - "/var/log/traefik/:/var/log/"
    deploy:
      replicas: 1
      labels:
        # enable Traefik for this service
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-net"

        # dashboard
        - "traefik.http.routers.traefik.rule=Host(`traefik.company.local`)"
        - "traefik.http.routers.traefik.entrypoints=traefik"
        - "traefik.http.routers.traefik.tls=true"
        - "traefik.http.routers.traefik.service=api@internal"

        # service
        - "traefik.http.services.traefik.loadbalancer.server.port=8080"
        - "traefik.http.services.traefik.loadbalancer.server.scheme=https"

networks:
  traefik-net:
    external: true
    name: traefik-net

traefik.yml:

providers:
  file:
    directory: "/etc/traefik"
    watch: true
  docker:
    swarmMode: true
    exposedByDefault: false

entrypoints:
  traefik:
    address: ":9000"
  backend:
    address: ":8082"
  backend-thrift:
    address: ":8092"
  backend-thrift-tcp:
    address: ":8093"

dynamic_conf.yml:

tls:
  stores:
    default:
      defaultCertificate:
        certFile: /cert/star.company.local.crt
        keyFile: /cert/star.company.local.key
  certificates:
    - certFile: /cert/star.company.local.crt
      keyFile: /cert/star.company.local.key
      stores:
        - default
  options:
    default:
      minVersion: VersionTLS12

http:
  routers:
    catch-all-fallback:
      rule: "HostRegexp(`{host:(app1|app2|app3)(-(dev|test|stage))?\\.company\\.(local|com)}`)"
      priority: 1
      middlewares:
        - "redirect-to-generic-error-page"
      tls: {}
      service: "noop@internal"
  middlewares:
    redirect-to-generic-error-page:
      redirectRegex:
        regex: "(.*)"
        replacement: "https://www.company.com/maintenance.html?returnURL=${1}"
        permanent: false

Then, we deployed the TCP service (application) with the following configuration:

# docker-compose -f docker-compose-Prod.yml up
version: "3.7"
services:
  web:
    image: app:1.2.3
    networks:
      - traefik-net
    deploy:
      replicas: 1
      labels:
        # enable Traefik for this service
        - "traefik.enable=true"
        - "traefik.docker.network=traefik-net"

        # router
        - "traefik.http.routers.app-prod.tls=false"
        - "traefik.http.routers.app-prod.service=app-prod"
        - "traefik.http.routers.app-prod.entrypoints=backend"
        
        - "traefik.http.routers.app-prod-thrift.tls=false"
        - "traefik.http.routers.app-prod-thrift.service=app-prod-thrift"
        - "traefik.http.routers.app-prod-thrift.entrypoints=backend-thrift"

        - "traefik.tcp.routers.app-prod-thrift-tcp.tls=false"
        - "traefik.tcp.routers.app-prod-thrift-tcp.service=app-prod-thrift-tcp"
        - "traefik.tcp.routers.app-prod-thrift-tcp.entrypoints=backend-thrift-tcp"
        - "traefik.tcp.routers.app-prod-thrift-tcp.rule=HostSNI(`*`)"

        # service
        - "traefik.http.services.app-prod.loadbalancer.server.scheme=http"
        - "traefik.http.services.app-prod.loadbalancer.server.port=8082"

        - "traefik.http.services.app-prod-thrift.loadbalancer.server.scheme=http"
        - "traefik.http.services.app-prod-thrift.loadbalancer.server.port=8092"

        - "traefik.tcp.services.app-prod-thrift-tcp.loadbalancer.server.port=8093"
      restart_policy:
        condition: on-failure

networks:
  traefik-net:
    external: true
    name: traefik-net

What did you see instead?

When we send a TCP request to <host_IP>:8093 (entrypoint backend-thrift-tcp), we can see the following error in Traefik logs:

level=debug msg="http: TLS handshake error from 172.29.10.227:62608: tls: unsupported SSLv2 handshake received"

Why does Traefik even try to establish a TLS connection even though we explicitly told it not to with the following line?

traefik.tcp.routers.app-prod-thrift-tcp.tls=false

Interestingly, if we remove tls: {} line from catch-all-fallback HTTP router defined in static configuration, everything works fine.

What version of Traefik are you using?

Version:      2.9.10
Codename:     banon
Go version:   go1.20.3
Built:        2023-04-06T16:15:08Z
OS/Arch:      linux/amd64
2

is dynamic configuration. It needs to go into a dynamic config file which has to be loaded in static config via provider.file.

3

It indeed is placed in the dynamic configuration. I made a mistake in my original question, I edited it now. The router and middleware are displayed in Traefik dashboard so the problem lies somewhere else.

4

Ok, but the entrypoints need to go into static config.

5

I updated the question with all my configuration files for Traefik:

  1. docker-compose-Traefik.yml which contains a definition for deploying Traefik Docker service
  2. traefik.yml which contains static configuration
  3. dynamic_conf.yml which contains dynamic configuration

The last two files are mounted to Traefik container inside volumes section in docker-compose-Traefik.yml (volume ./TraefikConfig/:/etc/traefik/).

6

Sorry to be PITA again, but http.routers should go into dynamic config.

To be clean you can use provider.file just with the dynamic file, not the full folder. Static file is read automatically.

Check Traefik dashboard and debug log for errors.

7

You are correct, http.routers section should be placed in the dynamic configuration (dynamic_conf.yml). I updated the question with (I hope) correct configuration.

However, Traefik still wants to perform SSL handshake even if TLS for TCP router app-prod-thrift-tcp is set to false. If we remove tls: {} from catch-all-fallback router defined in dynamic_conf.yml, everything works okay.

Traefik dashboard doesn't show any errors. We can see the following error in Traefik log:

level=debug msg="http: TLS handshake error from 172.29.10.227:62608: tls: unsupported SSLv2 handshake received"

I don't understand why tls: {} in catch-all-fallback router has an effect on app-prod-thrift-tcp router. Is it because they listen on the same entrypoints? Is it a bug?

8

catch-all-fallback is not assigned to any entrypoint, so it is automatically assigned to all entrypoints and you then enable TLS.

Maybe try assigning it only to relevant entrypoints and if you want to use TLS and non-TLS, then use two catch-all, only one with TLS enabled.

9

Yes, catch-all-fallback automatically listens on all entrypoints, including the one that app-prod-thrift-tcp listens on. This is also visible in the Traefik dashboard.

However, I don't understand why a TLS setting of HTTP router (catch-all-fallback) has effect on TCP router (app-prod-thrift-tcp)? Is it because they use the same entrypoint? Is this documented somewhere?