Docker compose, bridge and host networking, and traefik puzzle

I am trying to create a setup using docker compose where I run

An example extract from my docker compose file is:

networks:
  rna-docker-exposed:
    external: true # means it is a fixed docker network created with "docker network create rna-docker-exposed"
    name: rna-docker-exposed # docker create network rna-docker-exposed
  rna-docker-nonexposed:
    internal: true # means it gets created especially for this compose and is called <dirname>_rna-docker-nonexposed
    driver: bridge

  dockerproxy: # see https://github.com/Tecnativa/docker-socket-proxy
    image: ghcr.io/tecnativa/docker-socket-proxy:0.1.1
    restart: unless-stopped
    mem_limit: 2G
    cpus: 0.75
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro   # never expose this container to the internet!
    environment:
      - CONTAINERS=1
      - LOG_LEVEL=debug
    networks:
      - rna-docker-nonexposed # use only internal network

  traefik:
    container_name: rnaserver-traefik
    restart: unless-stopped
    read_only: true
    mem_limit: 2G
    cpus: 0.75
    depends_on:
      - dockerproxy
    security_opt:
      - no-new-privileges:true
    image: traefik:v2.9.4
    user: 115:120
    ports:
      - "80:10080"  # high nr so we don't need to be root to bind
      - "443:10443" # ditto
    labels:
      - "traefik.enable=true"
      # Configure Traefik dashboard & api on secure entrypoint (":443"), for local LAN clients only
      - "traefik.http.routers.traefik-dashboard.entrypoints=websecure"
      - "traefik.http.routers.traefik-dashboard.tls=true"
      - "traefik.http.routers.traefik-dashboard.rule=Host(`foo.rna.nl`) && ClientIP(`192.168.2.1/24`) && (
PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      - "traefik.http.routers.traefik-dashboard.service=api@internal"
      - "traefik.http.routers.traefik-dashboard.middlewares=simpleAuth@file,rnalanWhitelist@file" # double on IP whitelist, this and ClientIP ...
    networks:
      - rna-docker-exposed
      - rna-docker-nonexposed

  whoami:
    image: traefik/whoami
    container_name: rnaserver-whoami
    restart: unless-stopped
    user: 117:122
    depends_on:
      - traefik
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=rna-docker-exposed"
      - "traefik.http.routers.whoami.rule=Host(`foo.rna.nl`) && PathPrefix(`/whoami`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls=true"
    networks:
      - rna-docker-exposed

Now this works. But I also want to run a service that – when it gets a connection from the outside world — doesn't think all traffic comes from docker, not from the local LAN. E.g. `traefik/whoami' now reports:

Hostname: 058aef33472f
IP: 127.0.0.1
IP: 172.23.0.4
RemoteAddr: 172.23.0.2:33464
GET /whoami HTTP/1.1
Host: foo.rna.nl
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.1 Safari/605.1.15
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9
X-Forwarded-For: 192.168.2.86
X-Forwarded-Host: foo.rna.nl
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: c13bfbe1e443
X-Real-Ip: 192.168.2.86

But I want it to report: RemoteAddr: 192.168.2.86:something

Can I combine

  • traefik not running as root, but used to route anyway (as now)
  • have a service that listens not on a bridge network, but on the host network?

My guess is not, because it would require traefik to be both network_mode: bridge and network_mode: host at the same time. But I might be missing something. Am I?

And suppose, I would like to change so that the/a whoami container reports RemoteAddr: 192.168.2.86:something outside of traefik. What options do I have? What about the TLS handling (that traefik now does for me)?

You want RemoteAddr to be the real IP of the client - I don’t think that is not possible with Traefik. The RemoteAddr is not a HTTP header but the info from the plain TCP/IP connection - and that one is coming from Traefik, which forwards the request.

So you final application should use the X-Real-IP header from HTTP instead of RemoteAddr of the connection.

What if the final application doesn't use HTTP (where you have the http header) but TCP? Or I have an application where I cannot influence what it uses, it doesn't look at X-Real-IP from traefik? Basically, I cannot change the using app, I need to solve this on the docker+traefik side, or keep this outside of traefik and/or docker.

In Docker, I can use a host network and I do not get the translation to docker's internal IP address range.

If you use plain TCP/IP (not HTTP), you could use a feature called ProxyProtocol. But like HTTP headers, your application needs to support it.

Otherwise it is technically not possible to get the external client IP when routing through Traefik (or any other reverse proxy).

If you need RemoteAddr to be the external client IP, then you need your service/container to listen directly on the external port.