[SOLVED] Does Traefik have a way to custom format the access log and get the Request_Id?

cakiwi · May 10, 2021, 1:44pm

Looks like you can leverage the tracing without a full jaeger setup. See this comment in github.

Add support for Correlation IDs

opened 08:45AM - 20 Mar 19 UTC

kind/enhancement kind/proposal

### Do you want to request a *feature* or report a *bug*? Feature ### What… did you expect to see? A few weeks ago, i created PR #4483 which adds support for Correlation IDs to Traefik which was closed. @juliens suggested that this might be worth discussing in a separate issue again, so here it is. To recap my reasoning for this feature request: > In our environment we make heavy use of Correlation IDs to track requests throughout our mircoservice infrastructure. We recently moved from Nginx to Traefik and it brought us a lot of benefits, but the possibility to add custom headers is somewhat limited compared to Nginx. We need to add a new random UUID to every requests that comes in from outside our infrastructure to properly track it's path through the environment. This could have been done with the help of custom request headers, but those have some disadvantages: > > If the header is already present, it will get overwritten. Also you cannot generate a new random UUID for every request. All you could do is generate a random UUID for every frontend each time Traefik reloads the configuration ( with the help of the spring template library ) but then you can have a lot of requests with the same correlation id. This is why we wrote a [http middleware](https://gitlab.com/JanMa/correlation) that adds these headers to every incoming request and would like to add it here. There has already been PR #2290 that addressed a similar issue but it was closed at the time in favor of supporting OpenTracing. Seeing that support for OpenTracing is now present in Traefik I would like to re-open the discussion of adding Correlation ID support to Traefik. > In my opinion the use case for a correlation id differs quite a lot from using a tracing engine. With correlation ids it is very easy to track an external request through all service calls in your environment. You "just" have to make sure your applications log the corresponding header name in their access logs and can easily gather all relevant log entries in a tool like Kibana. Having this information in your logging stack is incredibly useful for debugging, especially in a microservice environment. Using a tracing engine on the other hand is useful when you want to gather low level insights of your inter-service communication and see for example how long single remote procedure call takes. Also implementing full tracing support in all your services takes a lot more effort. I think having both features available in Traefik would be very beneficial. Please let me know what your thoughts are about this :-) Kind regards, Jan

edit:
Here is a json access log and the header as received by a traefik/whoami service

traefik_1 | {"ClientAddr":"172.18.0.1:49866","ClientHost":"172.18.0.1","ClientPort":"49866","ClientUsername":"-","DownstreamContentSize":413,"DownstreamStatus":200,"Duration":39971759,"OriginContentSize":413,"OriginDuration":39636734,"OriginStatus":200,"Overhead":335025,"RequestAddr":"localhost","RequestContentSize":0,"RequestCount":2,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/whoami","RequestPort":"-","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"RouterName":"whoami@docker","ServiceAddr":"172.18.0.3:80","ServiceName":"whoami-10571@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.18.0.3:80","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2021-05-10T13:52:16.918115828Z","StartUTC":"2021-05-10T13:52:16.918115828Z","downstream_Content-Length":"413","downstream_Content-Type":"text/plain; charset=utf-8","downstream_Date":"Mon, 10 May 2021 13:52:16 GMT","entryPointName":"web","level":"info","msg":"","origin_Content-Length":"413","origin_Content-Type":"text/plain; charset=utf-8","origin_Date":"Mon, 10 May 2021 13:52:16 GMT","request_Accept":"/","request_User-Agent":"curl/7.68.0","request_X-Forwarded-Host":"localhost","request_X-Forwarded-Port":"80","request_X-Forwarded-Proto":"http","request_X-Forwarded-Server":"77cbda8be111","request_X-Real-Ip":"172.18.0.1","request_X-Request-Id":"5f67d067566073cf:39dd5a59db9a5ed0:5f67d067566073cf:0","time":"2021-05-10T13:52:16Z"}

curl localhost/whoami
Hostname: 4424944b2024
IP: 127.0.0.1
IP: 172.18.0.3
RemoteAddr: 172.18.0.2:57996
GET /whoami HTTP/1.1
Host: localhost
User-Agent: curl/7.68.0
Accept: /
Accept-Encoding: gzip
X-Forwarded-For: 172.18.0.1
X-Forwarded-Host: localhost
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: 77cbda8be111
X-Real-Ip: 172.18.0.1
X-Request-Id: 5f67d067566073cf:39dd5a59db9a5ed0:5f67d067566073cf:0

Topic		Replies	Views
Access Logs - pass custom headers Traefik v2 docker , file	5	1183	May 11, 2023
Traefik access log Traefik v3 (latest) kubernetes-ingress	1	127	November 21, 2024
Regex in access log headers Traefik v2 kubernetes-ingress	1	592	May 11, 2021
Traefik Access logs showing on stdout but not in access.log Traefik v2 kubernetes-ingress	2	1987	December 21, 2022
Response Header Logging Traefik v2 kubernetes-crd	0	1728	May 18, 2022

[SOLVED] Does Traefik have a way to custom format the access log and get the Request_Id?

Related topics