Wrong IP picked by X-Real-IP from X-Forwarded-For IPs

Traefik Traefik v2
1

Issue: X-Real-IP is set to an internal IP and not to the client's IP, even though the client IP is listed in X-Forwarded-For header.

To illustrate the issue, I use the following setup: client/browser => Azure AppGateway => K8S (Azure AKS) => Traefik service => whoami service

Here are the headers that "whoami" service receives:

Hostname: whoami-5dfdf459f4-z9hl7
IP: 127.0.0.1
IP: ::1
IP: 10.244.40.20
IP: fe80::a861:ebff:feda:2465
RemoteAddr: 10.244.40.109:54294
GET /foo HTTP/1.1
...
X-Forwarded-For: 62.164.32.43:59271, 10.244.40.1
X-Forwarded-Server: traefik-ff84978f4-2t9dn
X-Real-Ip: 10.244.40.1

Note how the X-Forwarded-For header contains the real client IP (62.164.32.43), but X-Real-Ip is not referring it.

I looked at various blogs/discussion and I seem to follow recommended guidance:

  • set forwardedHeaders.insecure and proxyProtocol.insecure
  • configured externalTrafficPolicy: Local (although it doesn't seem to make any difference whether I select Cluster or Local - the result is the same as above)

My setup: Azure AKS Kubernetes cluster, Traefik 2.10.6 configured as LoadBalancer service, Azure Application Gateway in front of the AKS cluster.

Is there anything I'm missing?

P.S.: FWIW we're transitioning from ingress-nginx to Traefik. With ingress-nginx setting X-Real-Ip works fine (same setup as above: same cluster, same infra) using the following config:

proxy_set_header X-Real-IP $http_x_forwarded_for;
1 Like
2

You mention ProxyProtocol, did you enable it in Azure AppGateway?

3

Not really, I just mentioned it, as I tried different Traefik settings (and combinations). AppGateway sets six X-Forwarded-* headers, incl. X-Forwarded-For in my setup.