Middleware forwardauth does not apply

Hello,
I'm trying to put together a POC for using Traefik as an authenticating reverse proxy, that routes to a different backend docker container based on the X-Forwarded-User header returned by the (forward auth) authentication service.

Based on docs, I've managed to cobble together a dynamic forward auth configuration that authenticates, and then adds in the X-Forwarded-User header, and routes to a fixed container (your basic whoami service).

I have another dynamic config that has Traefik route to different containers based on on the X-Forwarded-User header with the label "traefik.http.routers.user1.rule=Headers("X-Forwarded-User","user1")" on the container that should get user1 traffic.

But the fragmentary nature of Traefik 2.0 documentation is biting me, because I am having trouble figuring out how to merge them into a single configuration. It seems that I can do a 2 stage setup, where I have one entrypoint accept the original request, authenticate it, and then direct the request it to a second, internal, entrypoint. And this entrypoint has containers using the Headers() rule to handle routing.

Unfortunately, whether it is 2 stages or a single stage, I don't see any examples of how to setup a configuration that does forward authentication, and then subsequently routes based on a header injected by the forward auth middleware. And the nature of the docs makes it hard for me to understand the "big picture" of the configuration enough to build such a config from scratch.

Does anyone have a configuration that is similar to this, or can someone point me at docs that explain the relationships between the different configuration directives better? I love the power that Traefik provides, but I feel like I'm reduced to stumbling around with cutting and pasting configs.

Thanks!

Sorry, this is not going to be as helpful as you hope.

First of all, it looks to me that 1 stage approach won't work. According to diagram and explanation in the documentations, routers are kicked in first, and then middlewares. If you are inserting your header in a middleware (you do not mention how you do this), then you are already past the routing and do not have a chance to route again.

It sounds like by chaining two instances of traefik together (or even doing something really crazy like routing it back to the same instance of traefik, which I never tried) you should be able to acheive what you want. You will need to experiement with this a little bit. Which brings me to my next point.

It is simply not possible to provide an example for each use case a user can come up with. What you are doing is very specific, it is unlikely, that anyone did that before and shared - that's why you cannot find any ready made recipie. Traefik is very flexible but this comes with a price - it can do so many things, that in order for you to achieve anything specific, you'll have to configure it properly yourself, since it is not possible to provide every possible example configuration on earth.

I do not think, you will be successful in getting someone to write your very specific configuration for you (unless a kind soul shows up), but if you describe what you are trying to do exactly, with your detailed configuration, what error are you gettting, what you are expecting to see and what is not working, it's quite probable, that some could explain why that does not work and what might (or might not).

Good luck!

Which directives and which relationships are not clear? If you can tell me that I can give it a go in explaining that.

1 Like

Hello zespri,
Thanks for replying - based on the docs, I didn't see a way to do a 1 stage configuration either, but hoped maybe someone might have a deeper knowledge and know a way.

How about I just take this one stage at a time for a 2 stage config?

Right now, I'm just trying to figure out how to get a static configuration with forwardAuth working, and can't quite figure out the right syntax. Here is my current file YAML provider.

http:
  routers:
    router0:
      service: whoami
      rule: "Host(\"localhost\")"
      entrypoints: web
      middlewares:
      - authsvc
  middlewares:
    authsvc:
      forwardauth:
        trustForwardHeader: true
        authResponseHeaders: X-Forwarded-User
        address: "http://authsvc/"
        entrypoints: web
  services:
    whoami:
      loadBalancer:
        servers:
        - url: http://whoami/

I have a docker configuration that brings up a "whoami" container and an "authsvc" container. Both seem to be accessible from the traefik container, but I seem to be missing some config directives. Here is the output from the traefik container complaining about the authsvc middleware.

traefik     | time="2019-09-24T22:56:03Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Start TCP Server" entryPointName=traefik
traefik     | time="2019-09-24T22:56:03Z" level=info msg="Starting provider *file.Provider {\"watch\":true,\"filename\":\"/tmp/traefik.yaml\"}"
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Start TCP Server" entryPointName=web
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Configuration received from provider file: {\"http\":{\"routers\":{\"router0\":{\"middlewares\":[\"authsvc\"],\"service\":\"whoami\",\"rule\":\"Host(\\\"localhost\\\")\"}},\"middlewares\":{\"authsvc\":{}},\"services\":{\"whoami\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://whoami/\"}],\"passHostHeader\":false}}}},\"tcp\":{},\"tls\":{}}" providerName=file
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="No entryPoint defined for this router, using the default one(s) instead: [web traefik]" routerName=router0@file
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating middleware" entryPointName=web routerName=router0@file serviceName=whoami middlewareName=pipelining middlewareType=Pipelining
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=router0@file serviceName=whoami
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating server 0 http://whoami/" serverName=0 serviceName=whoami entryPointName=web routerName=router0@file
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Added outgoing tracing middleware whoami" middlewareType=TracingForwarder entryPointName=web routerName=router0@file middlewareName=tracing
traefik     | time="2019-09-24T22:56:03Z" level=error msg="middleware \"authsvc@file\" does not exist" entryPointName=web routerName=router0@file
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating middleware" middlewareType=Pipelining routerName=router0@file entryPointName=traefik serviceName=whoami middlewareName=pipelining
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating load-balancer" routerName=router0@file entryPointName=traefik serviceName=whoami
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating server 0 http://whoami/" routerName=router0@file entryPointName=traefik serviceName=whoami serverName=0
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Added outgoing tracing middleware whoami" routerName=router0@file middlewareName=tracing middlewareType=TracingForwarder entryPointName=traefik
traefik     | time="2019-09-24T22:56:03Z" level=error msg="middleware \"authsvc@file\" does not exist" routerName=router0@file entryPointName=traefik
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="Creating middleware" entryPointName=traefik middlewareName=traefik-internal-recovery middlewareType=Recovery
traefik     | time="2019-09-24T22:56:03Z" level=debug msg="No default certificate, generating one"

The middleware section on forwardAuth doesn't give any extra directives that need to be put in the file.

Spelling mistake? Should be second capital "A"?

2 Likes

As @zespri said: Yaml is case sensitive.

You have also some configuration errors:

http:
  routers:
    router0:
      service: whoami
      rule: "Host(`localhost`)"
      entrypoints: web
      middlewares:
        - authsvc

  middlewares:
    authsvc:
      forwardAuth:
        trustForwardHeader: true
        authResponseHeaders:
        - X-Forwarded-User
        address: "http://authsvc/"

  services:
    whoami:
      loadBalancer:
        servers:
          - url: http://whoami/
1 Like

Thank you @zespri and @ldez - those are silly mistakes and I should have noticed them. It now works as expected.

The second step would be to replace the "whoami" destination with another internal entrypoint. To do that, would I simply define another listener bound to localhost on a different port as a new entrypoint, and then have the "service" entry under router0 point to a URL which is simply that new port?
Or is there a syntax for just handing off to a new entrypoint without adding a trip through the TCP stack?

I invite to read the following doc to understand the concepts in Traefik:

In fact, I have already read both of those pages. They are very high level, and I don't see if there is a way to have the service at the end of a router be another entrypoint.

the word "entrypoint" in your sentences create ambiguity:

  • an entrypoint for traefik is the port exposed by Traefik
  • a service reference the servers not entrypoint

I think, but I'm not sure, that you are using the word entrypoint to speak about server URL.

  services:
    whoami:
      loadBalancer:
        servers:
          - url: http://whoami/

So I don't understand your goals, could create a small scheme?

Hello Fernandez,
Thank you for taking the time to respond.
Actually, I mean "entrypoint" in the precise meaning of the diagram and the documentation. The idea is to setup a traefik configuration with 2 endpoints, one endpoint uses a forwardAuth middleware to authenticate a request based on headers, and then forwards the request to a second endpoint, listening ona different port, which will then apply a different routing rule based on the headers returned by the forwardAuth middleware. The second endpoint is not accessible remotely, and only used to apply a second set of routing rules.
I've built a test configuration that has 2 endpoints:

  1. The first endpoint is on port 80, and applies a forwardAuth middleware and then forwards the request to localhost:10000
  2. The second endpoint listens on port 10000, and currently only route to a whoami container. The eventual goal is to have it use the docker dynamic provider and route based on the X-Forwarded-User header

The problem is that the routing to the second entrypoint doesn't seem to be working. I can reach the second endpoint if I point a browser at port 10000, but when I go to port 80, the forwardAuth gets applied, and then the logs show that it is being forwarded to localhost:10000. However it looks like the router1 rules on the 'authenticated' endpoint are not finding a match and returns a 404 error.

static config

entryPoints:
  authenticated:
    address: "127.0.0.1:10000"
  web:
    address: "127.0.0.1:80"
http:
  routers:
    router0:
      service: whoami
      rule: "Host(\"localhost\")"
      entryPoints: ['web']
      middlewares:
      - authsvc
    router1:
      service: whoami2
      entryPoints: ['authenticated']
      rule: "Host(\"localhost\")"
  middlewares:
    authsvc:
      forwardAuth:
        trustForwardHeader: true
        authResponseHeaders:
          - X-Forwarded-User
        address: "http://authsvc/"
  services:
    whoami:
      loadBalancer:
        servers:
        - url: "http://127.0.0.1:10000/"
    whoami2:
      loadBalancer:
        servers:
        - url: "http://whoami2/"

First curl command against the 'web' endpoint and log entries

curl -vv -b kbase_session=DELETED http://localhost/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.54.0
> Accept: */*
> Cookie: kbase_session=DELETED
> 
< HTTP/1.1 404 Not Found
< Content-Length: 19
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 26 Sep 2019 16:34:25 GMT
< X-Content-Type-Options: nosniff
< 
404 page not found
* Connection #0 to host localhost left intact

[log entries for this request]

authsvc_1   | ('Got a request with cookie ', DELETED')
authsvc_1   | [pid: 8|app: 0|req: 1/1] 172.30.0.4 () {52 vars in 667 bytes} [Thu Sep 26 16:34:22 2019] GET / => generated 2 bytes in 3263 msecs (HTTP/1.1 200) 3 headers in 104 bytes (1 switches on core 0)
traefik     | time="2019-09-26T16:34:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Cookie\":[\"kbase_session=DELETED\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Forwarded-User\":[\"sychan\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:35252\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik     | time="2019-09-26T16:34:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Cookie\":[\"kbase_session=DELETED\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Forwarded-User\":[\"sychan\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:35252\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://127.0.0.1:10000/"
traefik     | time="2019-09-26T16:34:25Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"Cookie\":[\"kbase_session=DELETED\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Forwarded-User\":[\"sychan\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:35252\",\"RequestURI\":\"/\",\"TLS\":null}"

Here is a curl command to the second endpoint, that just routes to whoami (for now)

curl -vv http://localhost:10000/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 10000 (#0)
> GET / HTTP/1.1
> Host: localhost:10000
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Length: 341
< Content-Type: text/plain; charset=utf-8
< Date: Thu, 26 Sep 2019 16:35:58 GMT
< 
Hostname: whoami2
IP: 127.0.0.1
IP: 172.30.0.5
RemoteAddr: 172.30.0.4:48152
GET / HTTP/1.1
Host: whoami2
User-Agent: curl/7.54.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 172.30.0.1
X-Forwarded-Host: localhost:10000
X-Forwarded-Port: 10000
X-Forwarded-Proto: http
X-Forwarded-Server: fcc6ba3f19f1
X-Real-Ip: 172.30.0.1

* Connection #0 to host localhost left intact

[log entries for this request]

traefik     | time="2019-09-26T16:35:58Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost:10000\"],\"X-Forwarded-Port\":[\"10000\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost:10000\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:56428\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik     | time="2019-09-26T16:35:58Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost:10000\"],\"X-Forwarded-Port\":[\"10000\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost:10000\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:56428\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://whoami2/"
traefik     | time="2019-09-26T16:35:58Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept\":[\"*/*\"],\"User-Agent\":[\"curl/7.54.0\"],\"X-Forwarded-Host\":[\"localhost:10000\"],\"X-Forwarded-Port\":[\"10000\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"fcc6ba3f19f1\"],\"X-Real-Ip\":[\"172.30.0.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"localhost:10000\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.30.0.1:56428\",\"RequestURI\":\"/\",\"TLS\":null}"
It looks like the router2 rules fail to match because the original port that it came in on was port 80, and not port 10000? Any ideas how I can get the forwarding to work and have the rules for endpoint "authenticated" match the request?

Just out of curiosity, what image are you using forauthsvc?

Also, what host header do you expect your localhost:10000 to receive after authentication?

traefik.yml (static configuration)

# static configurationn
entryPoints:
  authenticated:
    address: ":10000"
  web:
    address: ":80"

providers:
  file:
    filename: /dynamic.yml

/dynamic.yml (dynamic configuration)

# dynamic configuration
http:
  routers:
    router0:
      rule: "Host(`localhost`)"
      entryPoints: ['web']
      middlewares:
        - authsvc
      service: whoami2
    router1:
      rule: "Host(`localhost`)"
      entryPoints: 
        - authenticated
        - redirect
      service: whoami2 # it will never called because of the redirect
      
  middlewares:
    authsvc:
      forwardAuth:
        trustForwardHeader: true
        authResponseHeaders:
          - X-Forwarded-User
        address: "http://authsvc/"
    redirect:
      redirectRegex:
        regex: "^http://localhost:10000/(.*)"
        replacement: "http://localhost:80/${1}"

  services:
    whoami2:
      loadBalancer:
        servers:
        - url: "http://whoami2/"

@ldez Thanks for replying with a config.

I ended up trying a different approach that didn't use a forthwardAuth middleware, but instead directed any request without a specific cookie to a default container, which would validate the cookie, start a docker container with a rule to match the cookie, and then have the browser reload itself after 5 seconds. At that point the container would be up with the rule matcher and would take over servicing requests for any request with the cookie.

In case you're curious, I have a small repo with the POC. You won't be able to do much with it unless you have an account in KBase to get an auth cookie, but you can see the logic: