Hi,
I do have a situation to solve: real IP to final backend.
At the moment I do have this situation:
Client -> Cloudflare -> Traefik TCP (docker) -> Traefik HTTP (kubernetes) -> Apache
On the Apache access logs I get as IP the kubernetes' node IP, not the Client IP.
What's the best way to implement both TCP on docker, HTTP on kubernetes and Apache to register the proper Client IP? Is there a proper guide/walkthrough?
Thank you.
By default, a reverse proxy using http will include headers with the original IP. If you use plain TCP, I think the only option is ProxyProtocol on each sender and receiver side (which will prefix any TCP connection with the origination IP).
I tryied to add proxyProtocol to the TCP traefik, and it breaked the Cloudflare SSL handshake.
Without it, all of the proxy hops work, without the client IP.
" If you're the owner of this website:
It appears that the SSL configuration used is not compatible with Cloudflare. This could happen for a several reasons, including no shared cipher suites."
time="2024-04-15T18:20:28+02:00" level=debug msg="Handling TCP connection from 172.69.58.110:40934 to KubernetesIP:443"
time="2024-04-15T18:20:28+02:00" level=debug msg="Handling TCP connection from 172.70.231.67:16092 to KubernetesIP:443"
time="2024-04-15T18:20:29+02:00" level=debug msg="Error while setting TCP connection deadline: set tcp 192.168.4.3:33792: use of closed network connection"
time="2024-04-15T18:20:29+02:00" level=debug msg="Error while setting TCP connection deadline: set tcp 192.168.4.3:33794: use of closed network connection"
time="2024-04-15T18:20:29+02:00" level=debug msg="Handling TCP connection from 162.158.148.219:16606 to KubernetesIP:443"
time="2024-04-15T18:20:29+02:00" level=debug msg="Handling TCP connection from 162.158.148.218:9788 to KubernetesIP:443"
time="2024-04-15T18:20:29+02:00" level=debug msg="Error while setting TCP connection deadline: set tcp 192.168.4.3:33798: use of closed network connection"
time="2024-04-15T18:20:30+02:00" level=debug msg="Handling TCP connection from 162.158.129.199:18400 to KubernetesIP:443"
time="2024-04-15T18:20:32+02:00" level=debug msg="Handling TCP connection from 172.69.6.94:34286 to KubernetesIP:443"
time="2024-04-15T18:20:32+02:00" level=debug msg="Error while setting TCP connection deadline: set tcp 192.168.4.3:33802: use of closed network connection"
TCP settings (first Traefik instance, on docker):
tcp: #http:
routers:
homelab-rtr-https:
rule: "HostSNIRegexp(`domain1.com`, `domain2.com`, `{subdomain:[a-z0-9]+}.domain1.com`, `{subdomain:[a-z0-9]+}.domain2.com`)"
entryPoints:
- websecure
#middlewares:
# - middlewares-chain-no-auth
service: homelab-svc-https
tls:
passthrough: true
certResolver: dns-cloudflare
domains:
- main: "domain1.com"
sans:
- "*.domain1.com"
- main: "domain2.com"
sans:
- "*.domain2.com"
options: tls-opts@file
homelab-rtr-http:
rule: "HostSNIRegexp(`domain1.com`, `domain2.com`, `{subdomain:[a-z0-9]+}.domain1.com`, `{subdomain:[a-z0-9]+}.domain2.com`)"
entryPoints:
- web
#middlewares:
# - middlewares-chain-no-auth
service: homelab-svc-http
#tls:
# passthrough: true
# certResolver: dns-cloudflare
# options: tls-opts@file
services:
homelab-svc-https:
loadBalancer:
servers:
- address: "KubernetesIP:443"
proxyProtocol:
version: 2
homelab-svc-http:
loadBalancer:
servers:
- address: "KubernetesIP:80"
proxyProtocol:
version: 2
HTTP settings (second traefik instance, on kubernetes):
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: traefik-hosting1-https-redirect
namespace: hosting1
spec:
redirectScheme:
scheme: https
permanent: true
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: traefik-hosting1-security
namespace: hosting1
spec:
headers:
frameDeny: true
sslRedirect: true
browserXssFilter: true
contentTypeNosniff: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 31536000
customRequestHeaders:
X-Forwarded-Proto: "https"
---
apiVersion: traefik.io/v1alpha1
kind: ServersTransport
metadata:
name: traefik-hosting1-transport
namespace: hosting1
spec:
insecureSkipVerify: true
---
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:
name: traefik-hosting1-tlsoptions
namespace: hosting1
spec:
minVersion: VersionTLS12
cipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- TLS_AES_256_GCM_SHA384
- TLS_AES_128_GCM_SHA256
- TLS_CHACHA20_POLY1305_SHA256
- TLS_FALLBACK_SCSV
curvePreferences:
- CurveP521
- CurveP384
sniStrict: false
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-hosting1-websecure
namespace: hosting1
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: HostRegexp(`domain1.com`, `domain2.com`, `{subdomain:[a-z0-9]+}.domain1.com`, `{subdomain:[a-z0-9]+}.domain2.com`)
services:
- name: hosting1-svc
port: 443
serversTransport: traefik-hosting1-transport
middlewares:
- name: traefik-hosting1-security
tls:
secretName: wildcard-hosting
options:
name: traefik-hosting1-tlsoptions
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik-hosting1-web
namespace: hosting1
spec:
entryPoints:
- web
routes:
- kind: Rule
match: HostRegexp(`domain1.com`, `domain2.com`, `{subdomain:[a-z0-9]+}.domain1.com`, `{subdomain:[a-z0-9]+}.domain2.com`)
services:
- name: hosting1-svc
port: 80
middlewares:
- name: traefik-hosting1-https-redirect
Did you enable ProxyProtocol in CF?
I think I'm not able to use CF ProxyProtocol as I'm not on an Enterprise Plan.
As an aside, shouldn't the IP be one of cloudflare's IPs instead of the last traefik IP in the line? it seems like every step in the line (first traefik tcp proxy, second traefik http proxy) rewrites the IP.
Ok, solved:
- --entrypoints.websecure.forwardedHeaders.trustedIPs=<list of Cloudflare's IPs from https://www.cloudflare.com/ips-v4/#> to the first Traefik instanceThank you very much for your time!