Debugging routing with middlewares

Traefik Traefik v3 (latest)
1

Hello, is there any way for me to debug incoming requests and more importantly, the transformations it undergoes with middlewares and external services?

For example, I have a use-case that takes a certain route, /map-client/* and then rewrites that to /map/dist/* and sends that to an external service (thus overwriting the original hostname). Visually everything looks fine, but I'm getting a 500 error when I try to call the endpoint. However when I try to use the address it should resolve into, it works fine, which to me indicates that somewhere along the way, something doesn't work the way it should.

But there's no way for me to figure out (besides various trial and error) where that happens, as even the access logs don't contain this (I can see the request_X-Original-Url, request_X-Replaced-Path, curiously the RequestPath is the same as request_X-Original-Url).

Can anyone point me to a configuration or any other way to see what's going on? We're running Traefik 3.0.0 in Kubernetes via Helm chart.

2

By default Traefik sends the original host with the proxied request. Your external service probably expects their regular domain in request, try setting passHostHeader: false (doc).

3

Unfortunately, that didn't seem to work. I've tried setting up an in-memory Jaeger and enable tracing in Traefik. That showed me fortunately that the url.full attribute is correct, so my middlewares and external service looks correctly. Still the same response though.

I've tried getting Traefik to capture at least Host header, but to no avail. I've tried:

  • --tracing.capturedRequestHeaders[0]=Host as in docs
  • --tracing.capturedrequestheaders[0]=Host
  • --tracing.capturedrequestheaders=Host
  • env TRAEFIK_TRACING_CAPTUREDREQUESTHEADERS=Host

Nothing seems to work, in the logs where it shows the static configuration in the beginning, none of those seems parsed/recognized and the captured traces don't show this header.

4

You could target an "external" container like traefik/whoami (link), which will show you the headers. Instead of the original external target.

1 Like
5

Interestingly enough, --tracing.capturedRequestHeaders=Host seemed to work (it did show up in the static configuration), but it doesn't get sent to Jaeger as an additional argument (verified by tapping the jaeger-collector service with MITM proxy)

EDIT: The Host header specifically didn't get captured because of Go implementation of http.Request.Header:

For incoming requests, the Host header is promoted to the Request.Host field and removed from the Header map

Something like --tracing.capturedRequestHeaders=X-Forwarded-Host,X-Forwarded-For indeed does work.

6

Looks like the issue was in http/https mismatch (probably sending HTTP request to HTTPS protocol and port), because all headers seemed to be correct.

7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.