Auth headers not set (rootless docker + traefik + oauth2-proxy + keycloak)

Traefik Traefik v3 (latest)
,
1

Hi folks,
I'm having some really hard time configuring the stack mentioned in the object. Everything actually works quite well, except for the integration of forwardauth+oauth2-proxy to protect sensible endpoints. I have no idea if the issue is with oauth2-proxy or with the traefik forwardauth middleware, but I had to start somewhere :).

In short, I am running several services (including traefik) using rootless docker. My keycloak instance is working fine (other services with native Keycloak/OIDC support work flawlessly) and is hosted on, let's say, kc.mydomain.com. Now, I want to protect some services (e.g. mailu) which only support authentication through proxy-set headers. However, I really can't seem to find the correct configuration to actually set these headers!

The traefik/oauth2-proxy compose stack is as follows (I have added the whoami service for testing, as you will see):

docker-compose.yml

services:
  traefik:
    image: traefik:v3.6
    restart: always
    ports:
      - 80:8080
      - 443:8443
    networks:
      - default
      - oauth2-proxy
    volumes:
      - ./traefik.yml:/etc/traefik/traefik.yml
      - /run/user/1000/docker.sock:/var/run/docker.sock
      - ./dynamic:/dynamic
      - /data/traefik/acme.json:/acme.json
      - /data/logs/traefik:/var/log/traefik

  oauth2-proxy:
    image: quay.io/oauth2-proxy/oauth2-proxy:v7.14.3
    command: --config /oauth2-proxy.cfg
    volumes:
      - ./oauth2-proxy.cfg:/oauth2-proxy.cfg
    networks:
      - oauth2-proxy
    depends_on:
      - traefik
    environment:
      - OAUTH2_PROXY_CLIENT_SECRET=${OAUTH2_PROXY_CLIENT_SECRET}
      - OAUTH2_PROXY_COOKIE_SECRET=${OAUTH2_PROXY_COOKIE_SECRET}

  whoami:
    image: traefik/whoami
    networks:
      - default
    depends_on:
      - traefik

networks:
  default:
    name: reverse_proxy
  oauth2-proxy:
    driver: bridge

dynamic/oauth.yml

http:
  routers:
    oauth2-proxy:
      rule: "Host(`auth.mydomain.com`)"
      service: oauth2-proxy
      entrypoints: https
      middlewares:
        - "auth-headers@file"
      tls:
        certResolver: resolver

    whoami:
      rule: "Host(`whoami.mydomain.com`)"
      service: whoami
      entrypoints: https
      middlewares:
        - "oauth2@file"
      tls:
        certResolver: resolver

  services:
    oauth2-proxy:
      loadbalancer:
        servers:
          - url: "http://oauth2-proxy:4180"

    whoami:
      loadbalancer:
        servers:
          - url: "http://whoami:80"

  middlewares:
    oauth2:
      forwardauth:
        address: "https://auth.mydomain.com/oauth2/auth"
        authRequestHeaders:
          - "Cookie"
        authResponseHeaders:
          - "X-Forwarded-Email"
        trustForwardHeader: true

    auth-headers:
      headers:
        sslRedirect: true
        stsSeconds: 315360000
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        sslHost: mydomain.com
        stsIncludeSubdomains: true
        stsPreload: true
        frameDeny: true

oauth-proxy.cfg

http_address="0.0.0.0:4180"
skip_provider_button="true"
cookie_secure="true"
cookie_domains=["mydomain.com", "*.mydomain.com"]
whitelist_domains=["mydomain.com", "*.mydomain.com"]

# Traefik
reverse_proxy="true"
upstreams="static://202"

# Keycloak
provider="keycloak-oidc"
client_id = "oauth2-proxy"
redirect_url="https://auth.mydomain.com/oauth2/callback"
oidc_issuer_url="https://kc.mydomain.com/realms/myrealm"
email_domains="mydomain.com"
code_challenge_method="S256"

# Logging
auth_logging="true"
request_logging="true"
standard_logging="true"
silence_ping_logging="true"

I have tried any kind of possible combination of the authRequestHeaders and authResponseHeaders of the forwardauth middleware, as well as any settings I could think of for ouath2-proxy, but no matter what this is the data returned by the whoami service:

Hostname: 7ba59c58627b
IP: 127.0.0.1
IP: ::1
IP: 172.18.0.9
RemoteAddr: 172.18.0.2:43990
GET / HTTP/1.1
Host: whoami.mydomain.com
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:147.0) Gecko/20100101 Firefox/147.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding: gzip, deflate, br, zstd
Accept-Language: en-US,en;q=0.9
Cookie: oauth2_proxy_0=fvp2BX22qDotCdnYu-LXySkbU3Gk61b1BInhKEoeh2Gk4DO01E76h7GsTVIfXtoLcQT3WEBheUAkI8stceqsiwNZ5yeY1aQ0BWaqN6JXIOsrTFwQSuj-exwykp8MgHY-5mpK-J-QLr9qRdOieoOiIzfL0WcwYgiPYcey01ghn5HNGtMX6bHc3L_msEcRTvmWtkO2Oj4b1DxRvP8M7jRDUtN2DluSRIxiUlyxgEU-jEIkOo9wDDPW1zlBjZr0Iwm5bi1aupWD-........................
Priority: u=0, i
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Sec-Gpc: 1
Te: trailers
Upgrade-Insecure-Requests: 1
X-Forwarded-For: mydomain.com
X-Forwarded-Host: whoami.mydomain.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: 6fee585d8a55
X-Real-Ip: mydomain.com

So I always see the X-Forwarded-For/Host/Port/etc. headers, but nothing related to authentication. I can positively confirm that the authentication mechanism works, since I used the same strategy to protect the traefik dashboard. Moreover, if I visit the auth.mydomain/oauth2/userinfo page, I see all the information of logged in user. It just seems that that information is not passed back the the originally visited service through headers.

I'm quite stuck here, maybe somebody can help? Thanks!

2

Did you check the examples (compose, traefik, oauth2 files)?

3

Hi @bluepuma77 I did! The only sensible difference between my setup and what I found on the examples and documentation is that in my case I’m using rootless docker, but I’m not sure whether this is relevant to this issue.
Do you have any suggestion on how to verify if the problem is in oauth2-proxy or in forwardauth? I mean, is there some way to check the headers added by oauth2-proxy before passing through the forwardauth middleware?

Thanks!

4

Just a regular user, I am sure the rootless Docker has nothing to do with it.

Did you set the headers to be forwarded from oauth2-proxy to the target service in Traefik dynamic config (example)?

  middlewares:

    oauth-auth-redirect:
      forwardAuth:
        address: http://oauth2-proxy:4180
        trustForwardHeader: true
        authResponseHeaders:
          - X-Auth-Request-Access-Token
          - Authorization