X-Real-Ip header wrong

Traefik Traefik v3 (latest)
1

Hello everyone, I am running the latest version of traefik in docker, according to the recommendations I did in my compose.yml:

    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host

I also use cloudflare proxy and in this case all headers arrive as needed, real client addresses are substituted.

But if I turn off the proxy, then instead of real client addresses I see the addresses of my docker network

Hostname: 29dcd0dc9ea4
IP: 127.0.0.1
IP: ::1
IP: 192.168.147.15
IP: fe80::6439:79ff:feab:2763
RemoteAddr: 192.168.147.14:43230
GET / HTTP/1.1
Host: deleted
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: deleted
Cache-Control: max-age=0
Cookie: deleted 
Priority: u=0, i
Sec-Ch-Ua: "Not;A=Brand";v="99", "Google Chrome";v="139", "Chromium";v="139"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 192.168.147.14
X-Forwarded-Host: deleted
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 13372420a2d8
X-Real-Ip: 192.168.147.14

I tried all possible settings including

      - "--entryPoints.web.forwardedHeaders.insecure=true"
      - "--entryPoints.websecure.forwardedHeaders.insecure=true"
      - "--entryPoints.websecure.proxyProtocol.insecure=true"

But I can't solve the problem. The whoami service gives the client's address correctly only when you come through the cloudflare proxy, but if you disable proxying or register a real address in the hosts, everything breaks. Where should I look? I need help

2

If you want to use the headers from Cloudflare, you need to set them to trusted (doc).

3

Thanks for the answer, but I'm afraid you didn't read carefully, there are no problems with cloudflare, everything is perfect, I want to use it without it. And in this case everything breaks and I don't understand why. In fact, I remove only 1 proxy from the chain, nothing should change, but without proxying from cloudflare it breaks and with it everything shows correctly)))

4

Not sure what you are trying to do. You directly point your domain to your Traefik server IP?

For a single Docker container, you should get correct X-Real-IP when you just publish the port (example). For Swarm you need to explicitly set host mode, otherwise an ingress network is used.

5

Here is my compose file, I run containers without swarm

networks:
  ocstore_private_network:
    name: ocstore_private_network
    internal: true
    ipam:
      config:
        - subnet: 192.168.147.0/24
          gateway: 192.168.147.1
  ocstore_public_network:
    name: ocstore_public_network
    external: true
services:
  traefik:
    image: ${TRAEFIK_IMAGE_TAG}
    container_name: traefik
    command:
      - "--log.level=${TRAEFIK_LOG_LEVEL}"
      - "--log.filePath=/var/log/traefik/traefik.log"
      - "--accesslog=true"
      - "--accesslog.filepath=/var/log/traefik/access.log"
      - "--accessLog.format=json"
      - "--accessLog.fields.defaultmode=keep"
      - "--accessLog.fields.headers.defaultmode=keep"
      - "--accessLog.fields.headers.names.CF-Connecting-IP=keep"
      - "--api.dashboard=true"
      - "--api.insecure=true"
      - "--ping=true"
      - "--ping.entrypoint=ping"
      - "--entryPoints.ping.address=:8082"
      - "--entryPoints.web.address=:80"
      - "--entryPoints.web.forwardedHeaders.insecure=true"
      - "--entryPoints.websecure.forwardedHeaders.insecure=true"
      - "--entryPoints.websecure.proxyProtocol.insecure=true"
      - "--entryPoints.websecure.address=:443"
      - "--providers.docker=true"
      - "--providers.docker.endpoint=unix:///var/run/docker.sock"
      - "--providers.docker.exposedByDefault=false"
      - "--certificatesresolvers.cloudflare.acme.dnschallenge=true"
      - "--certificatesresolvers.cloudflare.acme.dnschallenge.propagation.delaybeforechecks=2"
      - "--certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare"
      - "--certificatesresolvers.cloudflare.acme.dnschallenge.resolvers=1.1.1.1:53,1.0.0.1:53"
      - "--certificatesresolvers.cloudflare.acme.email=${TRAEFIK_ACME_EMAIL}"
      - "--certificatesresolvers.cloudflare.acme.storage=/etc/traefik/acme/acme.json"
      - "--certificatesresolvers.cloudflare.acme.caServer=https://acme-v02.api.letsencrypt.org/directory"
      - "--metrics.prometheus=true"
      - "--metrics.prometheus.buckets=0.1,0.3,1.2,5.0"
      - "--global.checkNewVersion=true"
      - "--global.sendAnonymousUsage=false"
    environment:
      CF_DNS_API_TOKEN: ${CF_API_TOKEN}
      CF_ZONE_API_TOKEN: ${CF_API_TOKEN}
      CF_API_EMAIL: ${TRAEFIK_ACME_EMAIL}
    volumes:
      - /run/user/1000/podman/podman.sock:/var/run/docker.sock:ro
      - ./traefik/ssl:/etc/traefik/acme
      - ./traefik/logs:/var/log/traefik
      - /etc/localtime:/etc/localtime:ro
    networks:
      ocstore_private_network:
        ipv4_address: 192.168.147.14
      ocstore_public_network:
    ports:
      - target: 80
        published: 80
        mode: host
      - target: 443
        published: 443
        mode: host
      - target: 8080
        published: 8080
        mode: host
    healthcheck:
      test: ["CMD", "wget", "http://localhost:8082/ping", "--spider"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 5s
    labels:
      - "traefik.docker.network=ocstore_private_network"
      - "traefik.enable=true"
      - "traefik.http.routers.dashboard.rule=Host(`${TRAEFIK_HOSTNAME}`)"
      - "traefik.http.routers.dashboard.service=api@internal"
      - "traefik.http.routers.dashboard.entrypoints=websecure"
      - "traefik.http.services.dashboard.loadbalancer.server.port=8080"
      - "traefik.http.routers.dashboard.tls=true"
      - "traefik.http.routers.dashboard.tls.certresolver=cloudflare"
      - "traefik.http.routers.dashboard.tls.domains[0].main=my.domain"
      - "traefik.http.routers.dashboard.tls.domains[0].sans=*.my.domain"

      - "traefik.http.routers.dashboard.middlewares=authtraefik"
      - "traefik.http.middlewares.authtraefik.basicauth.users=${TRAEFIK_BASIC_AUTH}"
      - "traefik.http.routers.http-catchall.rule=HostRegexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
    restart: always

  whoami:
    image: traefik/whoami
    restart: always
    container_name: whoami
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`w.my.domain`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls=true"
    networks:
      ocstore_private_network:
        ipv4_address: 192.168.147.15

As you can see, I followed all the recommendations for this problem, I launched traefik and published its ports in host mode, I also added 2 networks, one internal without Internet access, the other with the external type, which has Internet access, traefik can see services in two networks. If the A records of my domain point to cloudflare servers, that is, the proxy mode is enabled in the DNS settings, then I get the correct display of the real IP of the client. If in the A record or hosts file on the remote client I explicitly specify the IP address of my domain, then the real address of the client is determined as the address of the internal docker network. And I do not understand this behavior. Because in the first case there is another proxy from cloudflare in front of my domain, and in the second case this proxy is not there, and everything in theory should work the same, but no!

whoami response output without Cloudflare Proxy mode

Hostname: dc6cae0dd98c
IP: 127.0.0.1
IP: ::1
IP: 10.89.1.13
IP: fe80::2422:5bff:feb0:d811
RemoteAddr: 10.89.1.14:47582
GET / HTTP/1.1
Host: w.my.domain
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br, zstd
Priority: u=0, i
Sec-Ch-Ua: "Not;A=Brand";v="99", "Google Chrome";v="139", "Chromium";v="139"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.89.1.14
X-Forwarded-Host: w.my.domain
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: e3b961097f61
X-Real-Ip: 10.89.1.14

X-Real-Ip: 10.89.1.14 - this address of Traefik container in ocstore_public_network
6

Output with Cloudflare Proxy mode ON

Hostname: f3d7f35f7e87
IP: 127.0.0.1
IP: ::1
IP: 10.89.0.3
IP: fe80::482a:76ff:fee9:8764
RemoteAddr: 10.89.0.6:56382
GET / HTTP/1.1
Host: w.my.domain
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, br
Cache-Control: max-age=0
Cdn-Loop: cloudflare; loops=1
Cf-Connecting-Ip: 77.77.77.77
Cf-Ipcountry: Country
Cf-Ray: RAY
Cf-Visitor: {"scheme":"https"}
Priority: u=0, i
Sec-Ch-Ua: "Not;A=Brand";v="99", "Google Chrome";v="139", "Chromium";v="139"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 77.77.77.77, 10.89.0.6
X-Forwarded-Host: w.my.domain
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 3cbd137df6f2
X-Real-Ip: 10.89.0.6

Here we can see the headers that Cloudflare sends:

  • Cf-Connecting-Ip: 77.77.77.77
  • Cf-Ipcountry: Country
  • Cf-Ray: RAY
  • Cf-Visitor: {"scheme":"https"}

And the correct header X-Forwarded-For: 77.77.77.77, 10.89.0.6
here I intentionally changed my real address to 77.77.77.77

But the header X-Real-Ip: 10.89.0.6 still points to the traefik address in the docker network, and this is normal behavior. Since in the X-Forwarded-For header we have everything we need.

But if I turn off the proxy mode, of course, the headers from Cloudflare will disappear and the X-Forwarded-For header will contain only the IP address of the traefik container in the docker network.

The whole problem is that my end application receives addresses of the internal docker network and not real addresses if I turn off the proxy mode. But I have a service where I am forced not to use the proxy mode and this has become a problem. I hope I explained it clearly.

Response result Proxy Mode Off, A record set to directly ip addr on my server

Hostname: f3d7f35f7e87
IP: 127.0.0.1
IP: ::1
IP: 10.89.0.3
IP: fe80::482a:76ff:fee9:8764
RemoteAddr: 10.89.0.6:34278
GET / HTTP/1.1
Host: w.my.domain
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br, zstd
Cache-Control: max-age=0
Priority: u=0, i
Sec-Ch-Ua: "Not;A=Brand";v="99", "Google Chrome";v="139", "Chromium";v="139"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
X-Forwarded-For: 10.89.0.6
X-Forwarded-Host: w.my.domain
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 3cbd137df6f2
X-Real-Ip: 10.89.0.6
7

How do you run Traefik? With Docker-CE on Linux?

8

Alma Linux 9 5.14.0-570.33.2.el9_6.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Aug 14 07:37:35 EDT 2025 x86_64 x86_64 x86_64 GNU/Linux

podman version 5.4.0

9

Not sure if podman will pass the correct IP to the container or does some NAT‘ing. Maybe check with simple whoami container and published port if the correct source IP is shown.

10

Dude, I gave a bunch of examples above, thanks for your input.

11

Well, it works with Docker, so if it doesn’t work with podman, then podman may be the reason.