Is this a Docker or a Traefik bug | X-Forwarded-For and X-Real-Ip not correctly transmitted?

[wylie]

Hi,

I´m trying to set up that Traefik 1.7.12 is forwarding Headers X-Forwarded-For and X-Real-Ip with the origin Client IPs. Network in between is a Docker "driver: bridge" net. The only IP which I can see with Wireshark is the gateway IP x.x.x.1 of the Docker Network itself. What is wrong here? Any help is really appreciated.

Here are some snippets out of the traefik compose file, traefik.toml and docker-compose.yml of an web app (nginx) behind the proxy.

-----------------------------------------------compose file traefik--------------------------------------

version: '3'

services:
  app:
    image: traefik:alpine
    container_name: traefik
    restart: always
    ports:
      - xxx.xxx.xxx.xxx:80:80
      - xxx.xxx.xxx.xxx:443:443
    labels:
      - traefik.frontend.rule=Host:monitor.xxxx.xxx
      - traefik.port=8080    
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /var/data/traefik/traefik.log:/traefik.log
      - /var/data/config/traefik/traefik.toml:/traefik.toml:ro
      - /var/data/traefik/acme.json:/acme.json
    environment:
      - "TZ=Europe/Paris"
    networks:
      web:
        ipv4_address: 172.28.255.254  #fix ipv4_address in docker bridge network web, so that containers in behind can trust traefik ipv4_address

networks:
  web:
    external:
      name: web

-----------------------------------------------traefik toml--------------------------------------

debug = false

logLevel = "ERROR"

defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.dashboard]
    address = ":8080"
    [entryPoints.dashboard.auth]
      [entryPoints.dashboard.auth.basic]
        users = ["admin:$xxx/"]
  [entryPoints.http]
    address = ":80"
      [entryPoints.http.redirect]
        entryPoint = "https"
  [entryPoints.https]
    address = ":443"
      [entryPoints.https.tls]

[traefikLog]
filePath = "/traefik.log"
format   = "json"

[api]
entrypoint="dashboard"

[acme]
email = "xxxxxx@xxxx.xxx"
storage = "acme.json"
entryPoint = "https"
onHostRule = true
  [acme.httpChallenge]
  entryPoint = "http"

[docker]
domain = "xxxxxx.xxx"
watch = true
network = "web"

-----------------------------------------------compose file web app--------------------------------------

web:
    image: nginx:alpine
    restart: always
    labels:
      - traefik.enable=true
      - traefik.backend=xxx
      - traefik.frontend.rule=Host:xxxx.xxxxxx.xxx
      - traefik.port=80
      - traefik.docker.network=web
      - traefik.frontend.redirect.permanent=true
    ports:
      - 19080:80
    links:
      - app
    volumes:
      - appvol:/var/www/html
    networks:
      - web

Cheers
mc

[james.dean]

No bug, use mode: host

[wylie]

Thank you for the hint! But, I don’t want to install Docker swarm as a Single Node Cluster for getting correct client IPs into the Header. It’s a bit like taking a sledgehammer to crack a nut.

https://worp.one/getting-original-client-ip-from-a-request-in-docker-swarm/

Is there really no other workaround out there at the moment. What’s about Traefik 2.0? Is there the same constraint in common with Docker without swarm?

[zespri]

As far as I understand the advice James give is generic, it is not specific to swarm. It's just that examples given there are for swarm, but you should be able to do the same without.

[wylie]

Thanks zespri! As I understand, host mode in a non docker swarm can't be used with containers which are in docker networks.

Traefik as a proxy for Docker container with host machines network

In swarm mode, host mode hasn't this constraint, because there is a additional overlay network which connects host network as well towards containers in docker networks. But, there are other constraints in terms of high availability.

https://www.quora.com/Whats-the-difference-between-Docker-Swarm-Docker-Compose-and-Docker-Networks

[zespri]

@wylie

Thanks zespri! As I understand, host mode in a non docker swarm can't be used with containers which are in docker networks.

Can you point me to where you get this from. I skimmed through the link you gave, and even searched for "host" on that page, but could not find anything that would indicate it. Of course on swarm the host mode is trickier, because there is more than one host - that's what that link says, but that certainly should not preclude single-host no-swarm setup.

[wylie]

At stackoverflow "Traefik as a proxy for Docker container with host machines network" user BMitch replies with "Without a common docker network, traefik won't be able to route to your container."

Regarding swarm mode constraints this will be discussed at before posted Worp.one link.

Quora.com link explains differences between docker-compose and swarm in general.

[zespri]

@wylie

"Without a common docker network, traefik won't be able to route to your container."

Yes, "common" here means common between traefik container and backend container. Host one should work in this case.

[wylie]

Okay, now I see where you're going with this. Basically it will work with host mode, but then all containers have to be run in host mode I guess and that's simply not possible in a complex docker environment. With swarm this would work, because you can still use docker networks with the Traefik running in host mode. But as already written I don't want to use swarm on a single host, because in my eyes this is a sledgehammer method and brings unnecessary complexity to - KISS - keep it stupid simple.

[wylie]

My question if this is a Docker or Traefik bug has still not been answered. Does nobody really know?

[zespri]

I don't think it's a bug per se, we just explored, how this happening above. That's just how the networking infrastructure built.

Traefik cannot inject this header, because it does not have access to the source IP. Docker cannot inject this header because it's not working with HTTP(s) protocol at all it has no visibility to these headers on the network layer. Do we have to assign blame?

[wylie]

I understand the background a little better now, thank you. So it's not a bug, but rather a limitation, or rather a missing possibility in today's technical standards.

Maybe it helps to think laterally. What about the 2.0 version? Can it also handle pure TCP routing? If yes, you could start here, if not it would be maybe useful to push this further.

My point here is in the least to accuse anyone. For me it is only important to communicate clearly what is possible and what is not, so that you don't end up in a dead end and in the worst case if you have already invested weeks or months in time.

[mr_vadim]

thanks bro, it's helps

[grasshide]

Hi there, since this is the first hit on google, I wanted to ask again if there is still no solution to this?
I also use a single host docker machine with a lot of containers that are reachable via traefik. They all have their own docker networks (e.g. one for my private cloud, one for home automation, etc.). Using traefik in network mode host would completely break that and I'd have to expose all my services on my docker machine that I just managed to secure via authelia.
All my logs are more or less useless (because I only see the docker gateway IP) and I really would like to use projects like crowdsec.
The last entry here is from two years ago. Is there still no solution?

[sc-starman]

I searched everywhere and tried all solutions for a swarm-based environment but no luck.
Is there still no solution for this typical matter?