Log rotation and client identification

Traefik Traefik v3 (latest)
1

Hi, I wanted to ask some info regarding logs, currently I have the following configuration:

log:
  level: DEBUG
  format: json
  filePath: /var/log/traefik/traefik-mgmt-01.log

accesslog:
   format: json
   filePath: /var/log/traefik/traefik-mgmt-01.access
   fields:
    names:
      StartUTC: drop

I saw that the log files are broken down by days automatically in the following format, but the access logs are not:

-rw-r--r--. 1 user root 1263788 Jan 2 08:41 traefik-mgmt-01-2025-01-02T07-41-43.561.log.gz
-rw-r--r--r--. 1 user root 1401818 Jan 4 05:11 traefik-mgmt-01-2025-01-04T04-11-21.052.log.gz
-rw-r--r--r--. 1 user root 1699942 Jan 7 10:04 traefik-mgmt-01-2025-01-07T09-04-02.487.log.gz
-rw-r--r--r--. 1 user root 1518592 Jan 9 08:28 traefik-mgmt-01-2025-01-09T07-28-34.413.log.gz
-rw-r--r--r--. 1 user root 1583076 Jan 11 14:30 traefik-mgmt-01-2025-01-11T13-30-42.602.log.gz
-rw-r--r--r--. 1 user root 1575834 Jan 13 19:25 traefik-mgmt-01-2025-01-13T18-25-17.634.log.gz
-rw-r--r--r--. 1 user root 1440084 Jan 15 14:20 traefik-mgmt-01-2025-01-15T13-20-44.133.log.gz
-rw-r--r--r--. 1 user root 1470456 Jan 17 11:03 traefik-mgmt-01-2025-01-17T10-03-50.182.log.gz
-rw-r--r--r--. 1 user root 1564065 Jan 19 20:09 traefik-mgmt-01-2025-01-19T19-09-42.158.log.gz

so at the moment I ended up with a 10 gig file where even the grep operation is slow.

I created a python script, which based on the time field, created individual files for me divided by day.

so i wanted to ask the following:

  1. is it possible to change the log split, via the static conf, so that both access logs and normal logs are handled?
  2. possibly is it possible to lock the log split, so that it can be managed externally with logrotate?

Also, the access logs contain only the following fields:

{
   "ClientAddr":"10.XXX.XXX.XXX:59917",
   "ClientHost":"10.XXX.XXX.XXX",
   "ClientPort":"59917",
   "ClientUsername":"-",
   "DownstreamContentSize":26,
   "DownstreamStatus":200,
   "Duration":6241950,
   "OriginContentSize":26,
   "OriginDuration":5940277,
   "OriginStatus":200,
   "Overhead":301673,
   "RequestAddr":"it-asset.XXXX.it",
   "RequestContentSize":53,
   "RequestCount":4569,
   "RequestHost":"it-asset.XXXX.it",
   "RequestMethod":"POST",
   "RequestPath":"/api/fXXXXX/config",
   "RequestPort":"-",
   "RequestProtocol":"HTTP/1.1",
   "RequestScheme":"https",
   "RetryAttempts":0,
   "RouterName":"fleXXXXX-01@docker",
   "ServiceAddr":"172.XXX.XXX.9:8080",
   "ServiceName":"fleXXXXX-01@docker",
   "ServiceURL":"http://172.XXX.XXX.9:8080",
   "SpanId":"0000000000000000",
   "StartLocal":"2025-01-20T11:19:11.728821114+01:00",
   "TLSCipher":"TLS_AES_128_GCM_SHA256",
   "TLSVersion":"1.3",
   "TraceId":"00000000000000000000000000000000",
   "entryPointName":"websecure",
   "level":"info",
   "msg":"",
   "time":"2025-01-20T11:19:11+01:00"
}

is it possible to make sure that they include all the additional info, like client pc hostname, user username or any other info, which allows me to identify the user in case of access violation?

Thanks

2

Interesting, I didn't know that Traefik log has rotation (doc), it was introduced with v3. It seems Traefik access log currently does not support it natively (doc), except for USR1 signal.

HTTP requests come from an IP, so that's what Traefik can see. "client pc hostname" is not available. "username" is not available, unless the web server asked for dedicated authentication (browser pop-up for username/password).

3

okay, so how can I manage the logs, so that they are managed on a daily basis?

4

As the Traefik doc states, you need to manage it externally:

Log Rotation

Traefik will close and reopen its log files, assuming they're configured, on receipt of a USR1 signal. This allows the logs to be rotated and processed by an external program, such as logrotate.

And an Internet search for traefik logrotate will present solutions like this.

1 Like
5

As mentioned earlier, Traefik does not handle log rotation for access logs out of the box. To address this, I opted to run a separate Docker service within my stack specifically for log rotation. This allows me to manage log rotation independently, using a configuration tailored to my needs.

One effective solution is the samuelrunggaldier/logrotate Docker image. Here's a basic example of how you can integrate it into your Docker Swarm stack:

version: '3.8'

services:
  # Traefik
  traefik:
    image: traefik:v3.4.4
    environment:
      - TZ="Europe/Berlin"
    command:
      # Other configs

      # Logging (general)
      - "--log.level=ERROR"
      - "--log.filepath=/logs/traefik.log"
      - "--log.format=json"

      # Access log
      - "--accesslog.filepath=/logs/access.log"
      - "--accesslog.format=json"
      - "--accesslog.bufferingsize=100"
      - "--accesslog.fields.headers.defaultmode=keep"

      # Built-in log rotation disabled, use external logrotate service

    volumes:
      - traefik_logs:/logs

  # Logrotate service
  logrotate:
    image: samuelrunggaldier/logrotate:latest
    volumes:
      - traefik_logs:/logs
    environment:
      - TZ="Europe/Berlin"
      - LOGS_PATH="/logs/*.log"
      - TRIGGER_INTERVAL="daily"
      - MAX_SIZE="NONE"
      - MAX_BACKUPS="365"
    deploy:
      restart_policy:
        condition: on-failure
        delay: 5s

volumes:
  traefik_logs: