IngressRouteTCP + SSH passthrough + External IP

Traefik Traefik v2 (latest)
,
1

Hello!

I am trying to use routing to passthrough SSH connections to an external service. We are using Traefik successfully with other HTTPS and TCP, but SSH seems to be not working at all. I believe it is probably a misconfiguration, or a missing detail within the configurations I am using, so here they are.

We have the entrypoint configured to listen on port 2222 and see it successfully expose, but when testing netcat to the entrypoint, it times out. I have verified network policies allow connectivity, and we can successfully connect to the external service from inside the cluster. But it seems that I am unable to port-forward with the service, or use it as a passthrough.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: sftp
  namespace: my-ns
spec:
  entryPoints:
    - sftp
  routes:
    - match: HostSNI(`*`)
      priority: 1
      services:
        - name: sftp-server-external
          port: 2222
  tls:
    passthrough: true
---
apiVersion: v1
kind: Service
metadata:
  name: sftp-server-external
  namespace: my-ns
  labels:
    kubernetes.io/service-name: sftp-server-external
spec:
  ports:
    - name: sftp-server-external
      port: 2222
      targetPort: 22
      protocol: TCP
---
apiVersion: v1
kind: Endpoints
metadata:
  name: sftp-server-external
  namespace: my-ns
  labels:
    kubernetes.io/service-name: sftp-server-external
subsets:
  - addresses:
      - ip: XXX.XXX.XXX.XXX

    ports:
      - name: sftp-server-external
        port: 22

I see no errors in the logs, and show that it creates the entrypoint and passthrough:

traefik-6b59798dfd-9nz7z traefik time="2023-09-26T08:42:32Z" level=debug msg="Creating TCP server 0 at XXX.XXX.XXX.XXX:22" entryPointName=sftp routerName=sftp-673acf455cb2dab0b43a@kubernetescrd serviceName=sftp-673acf455cb2dab0b43a serverName=0
traefik-6b59798dfd-9nz7z traefik time="2023-09-26T08:42:32Z" level=debug msg="Adding Passthrough route for \"HostSNI(`*`)\"" entryPointName=sftp routerName=sftp-673acf455cb2dab0b43a@kubernetescrd

I have been trying ever possible combination of using ExternalName service (but we need to remap the target port to the destination 2222 -> 22). I have looked through all different posts and nothing seems to specifically address this type of setup. None of the documentation has this either.

Any guidance or help would be greatly appreciated.

2

No clue about k8s, but try to remove tls: passthrough: true. This usually enables TLS in Traefik (Docker), but you want a plain TCP entrypoint and forward instead.

3

@bluepuma77 I read this suggestion before in another thread, and had tried removing this parameter but got a flood of errors in Traefik logs:

level=error msg="Error while handling TCP connection: readfrom tcp XX.XX.XX.XX:54370->YYY.YY.YYY.YYY:22: read tcp XX.XX.XX.XX:2222->YY.YY.YY.YY:37218: read: connection reset by peer"
4

But SSH is not using TLS anyway. Is something else accessing the port?

5

@bluepuma77 I found part of the issue I was having was firewall related, and the tls: passthrough parameter was also a problem. With the tls parameter declared Traefik will always insert HTTP headers into a TCP connection. I have things actually working correctly, EXCEPT with the tls parameter missing, I am still seeing the flood of Handling/Error connections in the traefik log.

level=error msg="Error while handling TCP connection: readfrom tcp 10.20.1.23:47464->10.9.252.36:2222: read tcp 10.20.1.23:2222->10.20.1.105:14088: read: connection reset by peer"
level=debug msg="Handling TCP connection from 10.20.1.105:44132 to 10.9.252.36:2222"
level=debug msg="Handling TCP connection from 10.20.0.33:7022 to 10.9.252.36:2222"
level=debug msg="Error while setting TCP connection deadline: set tcp 10.20.1.23:36976: use of closed network connection"
level=error msg="Error while handling TCP connection: readfrom tcp 10.20.1.23:47488->10.9.252.36:2222: read tcp 10.20.1.23:2222->10.20.1.15:37403: read: connection reset by peer"
level=debug msg="Handling TCP connection from 10.20.1.15:49190 to 10.9.252.36:2222"
level=error msg="Error while handling TCP connection: readfrom tcp 10.20.1.23:47494->10.9.252.36:2222: read tcp 10.20.1.23:2222->10.20.1.44:4063: read: connection reset by peer"
level=debug msg="Handling TCP connection from 10.20.1.44:51212 to 10.9.252.36:2222"

Since its a kubernetes service, I believe the load balancers are health checking the backend and causing this. I am unable to find anything in regards to annotations for an Ingress or Traefik to deal with this behavior.