ForwardAuth authResponseHeaders are not being persisted

Traefik Traefik v2
1

I'm trying to persist the "Remote-User" and "Remote-Groups" headers from Authelia's OK response to the request that's routed to my services. I'm seeing some weird behavior where the headers specified in authResponseHeaders ARE being persisted according to Traefik logs, but I'm not seeing them in the actual response.

My middlewares:

http:
  middlewares:
    middlewares-test-headers:
      headers:
        customResponseHeaders:
          Test-Header: "testing"
#          Remote-User: "somefakeuser"
#          Remote-Group: "somefakegroup"

    middlewares-authelia:
      forwardAuth:
        address: "http://authelia:9091/api/verify?rd=https://authelia.mydomain.com"
        trustForwardHeader: true
        authResponseHeaders:
          - "Remote-User"
          - "Remote-Groups"

Here are the headers I see in the browser:

Screenshot 2022-12-03 210312
Screenshot 2022-12-03 210312555×626 14.4 KB

Logs:

time="2022-12-03T21:00:07-06:00" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cookie\":[\"authelia_session=\"],\"Remote-Groups\":[\"correctremotegroup\"],\"Remote-User\":[\"correctremoteuser\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0\"],\"X-Forwarded-Host\":[\"rediscom.mydomain.com\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"aaf3e69bf504\"],\"X-Real-Ip\":[\"IP\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"rediscom.mydomain.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"IP\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2022-12-03T21:00:07-06:00" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://172.20.0.4:8081" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cookie\":[\"authelia_session=\"],\"Remote-Groups\":[\"admins\"],\"Remote-User\":[\"correctremoteuser\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0\"],\"X-Forwarded-Host\":[\"rediscom.mydomain.com\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"aaf3e69bf504\"],\"X-Real-Ip\":[\"IP\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"rediscom.mydomain.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"IP\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2022-12-03T21:00:07-06:00" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"OmitHost\":false,\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cookie\":[\"authelia_session=\"],\"Remote-Groups\":[\"correctremotegroup\"],\"Remote-User\":[\"correctremoteuser\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Te\":[\"trailers\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0\"],\"X-Forwarded-Host\":[\"rediscom.mydomain.com\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"aaf3e69bf504\"],\"X-Real-Ip\":[\"IP\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"rediscom.mydomain.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"IP\",\"RequestURI\":\"/\",\"TLS\":null}"

Interestingly, if I set them manually using the Headers middleware, they show up as expected.

I'm not sure what's going on, any help is appreciated. Might just be missing something obvious.

2

My understanding of the forward-auth is, that Traefik will send every request to auth first, then to the target service. So the authelia headers should reach your service, but I would not expect those in your response to the browser.

You can test with a whoami service to check if the headers arrive at your service.