TLS passthrough for HTTP router

Traefik Traefik v2
,
1

At home, I have HAProxy terminating TLS for some services running on my server.
What I want is to have a VPS with Traefik act as a front-end to my services hosted at home.
Basically, I am trying to build my own Cloudflare Tunnel without the downside of letting a third-party decrypt traffic.

I want connections to be
Internet --https--> Traefik on VPS (not decrypting traffic) --https--> HAProxy (decrypting traffic) --http--> services

So far,
Internet --https--> HAProxy (decrypting traffic) --http--> services
works well when whoami.mydomain points to HAProxy. The problem is on Traefik.

TCP router attempt

The only documented TLS passthrough option I see is for TCP routers. Since HTTPS uses TCP, I hope a TCP router can forward HTTPS traffic.

Config files

traefik.yml

entryPoints:
  websecure:
    address: ":443"

log:
  level: DEBUG

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    filename: /etc/traefik/dynamic_config.yml

certificatesresolvers:
  ...

dynamic_conf.yml

tcp:
  routers:
    whoami_rtr_tcp:
      rule: "HostSNI(`whoami.mydomain`)"
      service: svc_tcp
      tls:
        passthrough: true
  
  services:
    svc_tcp:
      loadBalancer:
        servers:
          - address: "<HAProxy_IP>:443"

Logs

But when I try to access my service, this is what I get

curl -v --resolve whoami.mydomain:443:<VPS_IP> https://whoami.mydomain
* Added whoami.mydomain:443:<VPS_IP> to DNS cache
* Hostname whoami.mydomain was found in DNS cache
*   Trying <VPS_IP>:443...
* Connected to whoami.mydomain (<VPS_IP>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
* error:0A000126:SSL routines::unexpected eof while reading
* Closing connection 0
curl: (35) error:0A000126:SSL routines::unexpected eof while reading

and stops there. The Traefik logs are

{"level":"debug","msg":"Handling TCP connection from <Client_IP>:56670 to <HAProxy_IP>:443","time":"2023-04-30T23:24:00+02:00"}
{"level":"error","msg":"Error while dialing backend: dial tcp <HAProxy_IP>:443: connect: connection timed out","time":"2023-04-30T23:26:00+02:00"}

and stop there.

Does anyone have an idea of how to do this, if it is even possible?

EDIT: I can access the service from the VPS with a curl command, so the problem is really with Traefik.

2

Use a plain TCP router with HostSNI(`*`) to forward all requests.

Traefik needs a cert to decrypt TLS to route by domain name.

3

Thanks for the reply @bluepuma77

Traefik needs a cert to decrypt TLS to route by domain name.

passthrough in the docs explicitly says it lets Traefik route without a cert (Traefik Routers Documentation - Traefik)

As seen above, a TLS router will terminate the TLS connection by default. However, the passthrough option can be specified to set whether the requests should be forwarded "as is", keeping all data encrypted.

But for some reason, I can't manage to pass HTTPS traffic via a TCP router.

The SNI says what domain is requested, so Traefik does not need to decrypt anything in order to forward traffic.

4

To my knowledge you need the cert to read the hostname from a TLS request. If you don’t have the cert in Traefik (custom or LE), you can only create a router with * domain to match and forward the request.

5

Ok, I have tried it

tcp:
  routers:
    whoami_rtr_tcp:
      rule: "HostSNI(`*`)"
      service: svc_tcp
      tls:
        passthrough: true
  
  services:
    svc_tcp:
      loadBalancer:
        servers:
          - address: "<HAProxy_IP>:443"

and the logs are similar on Traefik

time="2023-05-06T01:20:01+02:00" level=debug msg="Handling TCP connection from <Client_IP>:20523 to <HAProxy_IP>:443"
time="2023-05-06T01:22:10+02:00" level=error msg="Error while dialing backend: dial tcp <HAProxy_IP>:443: connect: connection timed out"

Do you have any idea how to make it work @bluepuma77?

6

Have you tried a wget to your target from within your Traefik container?

7

Yes, wget works from within Traefik.

In the meantime, I have setup a similar TCP router in HAProxy and it works.
I would prefer Traefik though.

8

Try removing the whole TLS config:

If it's a TCP router (without any TLS configured), it will just forward plain TCP.

1 Like
9

I removed the TLS config.

Device logs:

$ curl -v --resolve whoami.mydomain:443:<VPS_IP> "https://whoami.mydomain"
* Added whoami.mydomain:443:<VPS_IP> to DNS cache
* Hostname whoami.mydomain was found in DNS cache
*   Trying <VPS_IP>:443...
* Connected to whoami.mydomain (<VPS_IP>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* (5454) (IN), , Unknown (72):
* error:0A00010B:SSL routines::wrong version number
* Closing connection 0
curl: (35) error:0A00010B:SSL routines::wrong version number

Traefik logs

time="2023-05-10T03:32:08+02:00" level=debug msg="Handling TCP connection from <Client_IP>:55517 to <HAProxy_IP>:443"
time="2023-05-10T03:32:09+02:00" level=debug msg="Error while setting TCP connection deadline: set tcp 192.168.26.2:36650: use of closed network connection"

Now, with the TLS passthrough, for some reason I get the same curl error wrong version number instead of the previous one unexpected eof while reading.

10

Strange. Where is the haproxy cert coming from?

If your traffic is http, how about you use http entrypoints 80+443 on Traefik, use the certresolver with tlsChallenge and simply activate serversTransport.insecureSkipVerify globally (or assign to a service) to forward with https to haproxy (with unknown cert)?