Traefik with FTPS

abdelilahBOU · June 6, 2024, 4:58pm

hello, i m trying to migrate from HAProxy to Traefik, i have an existing config of Haproxy that act as TCP loadbalancer for an FTPS server. now i m trying to that same with traefik.
The FPTS server Operate on the following port 4113 for control and 4114-4120 to transfer Data
here the current config for my Traefik:

static Config
api:
  insecure: true
  dashboard: true

log:
  level: "DEBUG"
  filePath: "/logs/traefik.log"

certificatesResolvers:
  letsEncrypt:
    acme:
      email: "name@email"  # Email address used for registration
      storage: "/etc/traefik/acme/acme.json"    # File or key used for certificates storage
      tlsChallenge: {}


entryPoints:
  ftps:
    address: ":4113"
  ftps_data_1:
    address: ":4114"
  ftps_data_2:
    address: ":4115"
  ftps_data_3:
    address: ":4116"
  ftps_data_4:
    address: ":4117"
  ftps_data_5:
    address: ":4118"
  ftps_data_6:
    address: ":4119"
  ftps_data_7:
    address: ":4120"


metrics:
  prometheus: {}

providers:
  file:
    directory: "/etc/traefik/config"
    watch: true

    
serversTransport:
  insecureSkipVerify: true

Dynamic config
tcp:
  routers:
    hi_ftps_prod:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps"
        - "ftps_data_1"
        - "ftps_data_2"
        - "ftps_data_3"
        - "ftps_data_4"
        - "ftps_data_5"
        - "ftps_data_6"
        - "ftps_data_7"
      service: "ssl_hi_ftps_prod"
      tls: {}
    
    hi_ftps_stage:
      rule: "HostSNI(`ms-stage.hi.com`)"
      entryPoints:
        - "ftps"
        - "ftps_data_1"
        - "ftps_data_2"
        - "ftps_data_3"
        - "ftps_data_4"
        - "ftps_data_5"
        - "ftps_data_6"
        - "ftps_data_7"
      service: "ssl_hi_ftps_stage"
      tls: {}

  services: 
    ssl_hi_ftps_prod:
      loadBalancer:
        servers:
          - address: "172.20.149.86:4113" # Connectivity 
          - address: "172.20.149.86:4114" # Data 
          - address: "172.20.149.86:4115" # Data 
          - address: "172.20.149.86:4116" # Data 
          - address: "172.20.149.86:4117" # Data 
          - address: "172.20.149.86:4118" # Data 
          - address: "172.20.149.86:4119" # Data 
          - address: "172.20.149.86:4120" # Data  
    ssl_hi_ftps_stage:
      loadBalancer:
        servers:
        - address: "172.20.150.9:4113" # Connectivity 
        - address: "172.20.150.9:4114" # Data 
        - address: "172.20.150.9:4115" # Data
        - address: "172.20.150.9:4116" # Data 
        - address: "172.20.150.9:4117" # Data 
        - address: "172.20.150.9:4118" # Data 
        - address: "172.20.150.9:4119" # Data 
        - address: "172.20.150.9:4120" # Data

tls.yml
tls:
  certificates:
    - certFile: /etc/letsencrypt/certs/hi.com/fullchain.pem
      keyFile: /etc/letsencrypt/certs/hi.com/key.pem
  
  options:
    default:
      sniStrict: true

docker-compose.yml
services:
  traefik:
    image: traefik:2.10
    container_name: traefik
    network_mode: host
    restart: always
    volumes:
      - /srv/traefik/:/etc/traefik
      - /srv/traefik/logs:/logs
      - /etc/letsencrypt:/etc/letsencrypt:ro
    command: [ ]

from the logs i got:

time="" level=debug msg="Handling TCP connection from [my_ipv6]:40760 to 172.20.149.86:4114"
time="" level=error msg="Error while dialing backend: dial tcp 172.20.149.86:4114: connect: connection refused"
time="" level=debug msg="Handling TCP connection from [my_ipv6]:43232 to 172.20.149.86:4118"
time="" level=error msg="Error while dialing backend: dial tcp 172.20.149.86:4118: connect: connection refused"

i don't get it, i never setup such a thing with traefik , also when the setup was done with haproxy , it prompt me in filezilla to accept the certifcate when i try to connect to the ftps server.
anyone that have tried to do such a config kindly provide me with some guidance?

bluepuma77 · June 6, 2024, 6:02pm

Are those ports TCP or UDP?

It seems you didn’t open any ports on Traefik container.

You probably need to create 7 different routers/services because I don’t think your target server likes receiving packets on port 4114 when it was intended for 4120.

It might not work at all, as a chatbot suggests that the initial connection is TCP but not TLS. So regular HostSNI() would not work. You could try HostSNI(`*`), but then you can only have a single target on the port, as Traefik can’t differentiate.

Note: you can only use one static config, traefik.yml or command:.

abdelilahBOU · June 7, 2024, 8:44am

Thus are TCP Ports, the network_mode: host setting means that the container will share the host system's networking namespace.

abdelilahBOU · June 7, 2024, 9:35am

when using HostSNI('*')

time="" level=debug msg="Error while terminating TCP connection: tls: CloseWrite called before handshake complete"
time="" level=debug msg="Handling TCP connection from [MY_ip]:47696 to 172.20.149.86:4113"
time="" level=error msg="Error while handling TCP connection: readfrom tcp 172.20.148.13:53494->172.20.149.86:4113: tls: client requested unsupported application protocols ([ftp])"
time="" level=debug msg="Error while terminating TCP connection: tls: CloseWrite called before handshake complete"

bluepuma77 · June 7, 2024, 12:50pm

Under the assumption it's not a "real" TLS connection, you need to remove tls: {}, so no TLS is activated for the router and only plain TCP/IP is used to proxy the connection to the target service.

abdelilahBOU · June 9, 2024, 9:41am

i m in need to use TLS connection, but i would like the LB to give me his own certifcate to accept it with filezilla in other word, a secure connection between client and the LB, the the LB would re-transmit the trafiic to the backend.
i got some progess using the following config :

tcp:
  routers:
    ## Connect
    hi_ftps_prod_connect:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps"
      service: "ssl_hi_ftps_prod_connect"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_1:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_1"
      service: "ssl_hi_ftps_prod_passive_1"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_2:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_2"
      service: "ssl_hi_ftps_prod_passive_2"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_3:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_3"
      service: "ssl_hi_ftps_prod_passive_3"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_4:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_4"
      service: "ssl_hi_ftps_prod_passive_4"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_5:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_5"
      service: "ssl_hi_ftps_prod_passive_5"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_6:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_6"
      service: "ssl_hi_ftps_prod_passive_6"
      tls: true

    ## Data transfer
    hi_ftps_prod_passive_7:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_7"
      service: "ssl_hi_ftps_prod_passive_7"
      tls: true

    
  services:
    
    ## Prod Connect
    ssl_hi_ftps_prod_connect:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4243" # Connectivity 
        
    ## Data transfer
    ssl_hi_ftps_prod_passive_1:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4244" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_2:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4245" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_3:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4246" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_4:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4247" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_5:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4248" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_6:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4249" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_7:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4250" # Data

now i got from the logs

time="" level=error msg="Error while handling TCP connection: readfrom tcp MY-ip:48192->172.20.149.86:4243: tls: client requested unsupported application protocols ([ftp])"
time="" level=debug msg="Error while terminating TCP connection: tls: CloseWrite called before handshake complete"

NOTE: i m connecting with the Filezilla Client

bluepuma77 · June 9, 2024, 10:51am

As I wrote in an earlier post, it might not be a pure TLS connection:

There are reasons why FTP is considered a deprecated protocol. Have a look at WebDAV, which uses http, so should be reverse proxy compatible.

abdelilahBOU · June 11, 2024, 4:25pm

further information on the error with the ftp, i was messing the alpnProtocols on TLS handshake:
i should be something like the following :

tls:
  certificates:
    - certFile: /etc/letsencrypt/certs_files/fullchain.pem
      keyFile: /etc/letsencrypt/certs_files/key.pem
 
  options:
    default:
      alpnProtocols:
        - ftp

and the dynamic config will be :

tcp:
  routers:
    ## Connect
    hi_ftps_prod_connect:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps"
      service: "ssl_hi_ftps_prod_connect"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_1:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_1"
      service: "ssl_hi_ftps_prod_passive_1"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_2:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_2"
      service: "ssl_hi_ftps_prod_passive_2"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_3:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_3"
      service: "ssl_hi_ftps_prod_passive_3"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_4:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_4"
      service: "ssl_hi_ftps_prod_passive_4"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_5:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_5"
      service: "ssl_hi_ftps_prod_passive_5"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_6:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_6"
      service: "ssl_hi_ftps_prod_passive_6"
      tls: {}

    ## Data transfer
    hi_ftps_prod_passive_7:
      rule: "HostSNI(`ms.hi.com`)"
      entryPoints:
        - "ftps_data_7"
      service: "ssl_hi_ftps_prod_passive_7"
      tls: {}

    
  services:
    
    ## Prod Connect
    ssl_hi_ftps_prod_connect:
      loadBalancer:
        servers:
        - address: "172.20.149.86:4113" # Connectivity 
        
    ## Data transfer
    ssl_hi_ftps_prod_passive_1:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4114" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_2:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4115" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_3:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4116" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_4:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4117" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_5:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4118" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_6:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4119" # Data 

    ## Data transfer
    ssl_hi_ftps_prod_passive_7:
      loadBalancer:
        servers:
        - address: "172.20.49.6:4120" # Data

now i got the tls handsahke to complete between the FileZilla client and traefik , but i unable to get the directory from the server:
filezilla showig the following :

Command:	PWD
Response:	257 "/" is current directory.
Command:	TYPE I
Response:	200 Command TYPE okay.
Command:	EPSV
Response:	229 Entering Passive Mode (|||4114|)
Command:	MLSD
Response:	150 File status okay; about to open data connection.
Error:	Connection timed out after 20 seconds of inactivity
Error:	Failed to retrieve directory listing

logs from traefik :

time="# level=debug msg="Handling TCP connection from [MY-ip]:53050 to 172.20.49.6:4113"
time="#" level=debug msg="Handling TCP connection from [MY-ip]:50873 to 172.20.49.6:4114"

any idea what could be blocking traffic from going back to client ?

brablc · June 12, 2024, 9:40am

I am pretty new to Traefik, so sorry for blind shot, but I assume that neither HAProxy nor Traefik can terminate TLS from FTPS and you have to pass it thru:

    tls:
        passthrough: true

abdelilahBOU · June 12, 2024, 10:29am

it does terminate the TLS, with the last comment i mention, now i'm just facing a timeout from filezilla client when data transfer start.

Command:	PWD
Response:	257 "/" is current directory.
Command:	TYPE I
Response:	200 Command TYPE okay.
Command:	EPSV
Response:	229 Entering Passive Mode (|||4114|)
Command:	MLSD
Response:	150 File status okay; about to open data connection.
Error:	Connection timed out after 20 seconds of inactivity
Error:	Failed to retrieve directory listing

bluepuma77 · June 12, 2024, 11:27am

It’s probably an issue because you re-map the ports from internal to external to different port numbers.

The server doesn’t know about it and announces to the client certain ports to use, which are not available externally.

abdelilahBOU · September 6, 2024, 2:21pm

Guys, i m really struggling at this , i can't get data transfer, the control port works fine.
as recape here is dynamic routing:

tcp:
  routers:
    #FTPS PROD
    ftps_prod_control:
      entrypoints:
        - "ftps_control" # 1143
      rule: "HostSNI(`con-jo.per-cen.com`)"
      service: "ftps_prod_control"
      tls: {}
    
    ftps_prod_data1:
      entrypoints:
        - "ftps_data1" # 1144
      rule: "HostSNI(`con-jo.per-cen.com`)"
      service: "ftps_prod_data1"
      tls: {}
    
    ftps_prod_data2:
      entrypoints:
        - "ftps_data2" # 1145
      rule: "HostSNI(`con-jo.per-cen.com`)"
      service: "ftps_prod_data2"
      tls: {}

    ftps_prod_data3:
      entrypoints:
        - "ftps_data3" # 1146
      rule: "HostSNI(`con-jo.per-cen.com`)"
      service: "ftps_prod_data3"
      tls: {}

  services:
    # FTPS PROD
    ftps_prod_control:
      loadBalancer:
        servers:
          - address: "172.24.139.88:1143"

    ftps_prod_data1:
      loadBalancer:
        servers:
          - address: "172.24.139.88:1144"
    
    ftps_prod_data2:
      loadBalancer:
        servers:
          - address: "172.24.139.88:1145"

    ftps_prod_data3:
      loadBalancer:
        servers:
          - address: "172.24.139.88:1146"

tls.yml

tls:
  certificates:
    - certFile: /etc/letsencrypt/certs/fullchain.pem
      keyFile: /etc/letsencrypt/certs/key.pem

  options:
    default:
      sniStrict: false
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
      alpnprotocols:
        - http/1.1
        - h2
        - acme-tls/1
        - ftp

the issue still with DATA transfer:

Status:	Resolving address of con-jo.per-cen.com
Status:	Connecting to [IPV6]:4243...
Status:	Connection established, initializing TLS...
Status:	TLS connection established, waiting for welcome message...
Status:	Logged in
Status:	Retrieving directory listing...
Command:	PWD
Response:	257 "/" is current directory.
Command:	TYPE I
Response:	200 Command TYPE okay.
Command:	EPSV
Response:	229 Entering Passive Mode (|||1145|)
Command:	MLSD
Response:	150 File status okay; about to open data connection.
Error:	Connection timed out after 50 seconds of inactivity
Error:	Failed to retrieve directory listing

any help on waht could block data transfer

bluepuma77 · September 6, 2024, 3:51pm

In general FTPS a seen as a deprecated protocol. As you can see it’s not really made for modern Internet architecture, where connections are proxies around. Can’t you upgrade to something else?

SFTP s also not an option, as it uses no TLS at all, so you can only use 1 service per port (HostSNI(`*`)). But you could also use that for FTPS.

In general Traefik does terminate TLS, so you are aware that you proxy unencrypted, so the target service is plain FTP? Which announces the ports you are using?

A chatbot tells me that HostSNI is not used for the data connections. That doesn’t really make sense, but it could be one explanation for the issue.

Enable and check Traefik debug log and Traefik access log in JSON format.

Topic		Replies	Views
Haproxy in front of Traefik and Docker with Letsencrypt Traefik v2 docker , letsencrypt-acme	5	3842	February 21, 2023
Traefik and AWS Application Load Balancer integration Traefik v2 docker , letsencrypt-acme	0	1068	March 12, 2023
TLS 1.3 config for Traefik v1 Traefik v1 letsencrypt-acme	0	1053	October 18, 2019
Problem with Lets Encrypt. ACME Fails Traefik v2 docker , letsencrypt-acme	3	3191	March 11, 2022
ALPN Passthrough (2 tiered Traefik) Traefik v2 tcp	2	22	August 15, 2024

Traefik with FTPS

Related Topics