Separate access logs per service

In nginx we have an app log (for nginx itself), and then access and error logs per service. So app1 will have different logs to app2.

In traefik I managed to implement both the app log and access log. But the access log is the same for all services.

How can I get separate access logs per service?

According to access log doc this is not possible. You can use CLF and add RequestHost / RouterName or use JSON format and divide the logs in post-processing.

You could just ingest everything with (open source) solutions like Elastic/ELK or Grafana/Loki.

1 Like

Thanks. Added a feature request to the repo.

For anyone else who sees this, please upvote that issue if you also want this functionality.

Just for expectation management: I personally don’t see something like this implemented. It is very specific to the want of one user, no one else has asked for it. So far the Traefik team rather focused on more important issues.

Of course you could provide a pull request with the necessary code, either by implementing it yourself or hiring a freelance developer.

There are more that seem to want it. The the problem with the feature requests on the repo is that they are always closed and locked. That way no one can upvote them over time. So there is no way to know how many people want certain features. That's how it's done on most other repos.

I follow your development and I think I can already see your next feature request: log rotation :wink:

At the moment this can be done with signals (doc), but I don't think that's the way to handle it with Traefik in containers in a distributed system (Swarm/k8s).

But as stated before, feature requests sometimes don't get priority without a critical mass supporting it. Sometimes they are closed in favor of (kind of similar) existing issues. We can't read TLS certs via labels, can't set target URLs via labels, can`t get the proxy IP. (History)

1 Like

Hehe, thank you. I appreciate all the work you do for the traefik community! :pray: :smiley:

But actually I think log rotation must not be done inside almost any app - it should be out-of-process to avoid unnecessary disk access costs. Personally I already implemented that for traefik with logrotate, and it works very nicely.

(I'm not using swarm/k8s.)

BTW you are so knowledgeable for traefik, you should be working for the traefik company! They need you.

When a feature request is open we have 3 choices:

  • we think that is something that fits with Traefik then the issue is flagged as "enhancement"
  • we are not sure about the value or the traction then the issue is flagged as "proposal"
  • we think that doesn't fit with Traefik then the issue is closed and flagged as "declined"

The popularity of a declined issue will not drive us to implement it for several reasons, one of them is: if something is outside of the scope of Traefik, it's outside so we will not change our mind because of +1.

For example, Traefik is a reverse proxy and NOT a webserver, also Traefik is a reverse proxy for a cloud environment, in this context, it doesn't make sense to serve files. So we will not serve files even if there is a lot of +1.

The basic philosophy behind that is to handle 90% of the use cases, not 100%.
This is important to control the complexity and the maintainability of Traefik.

FYI: The company behind Traefik is Traefik Labs, and Traefik Labs is a small company.
Traefik Labs fired 25% of its employees in May 2023.
I was an employee in the engineering team and now I'm just a maintainer.

1 Like

FYI the log rotation is in the v3 https://github.com/traefik/traefik/pull/9515

2 Likes

I appreciate your words Idez, and I'm sorry for your troubles. I hope you go on to greater things.

PS, log rotation feature is nice but typically heavy for an app (logrotate is very light and out of process)... Compared to that, log splitting by vhost is light and more useful ad doesn't require complicated environment. But we can disagree and still be friends. :smiley:

I agree with you: log rotation is heavy and I recommend not using the internal log rotation even if I created it.

It's the same for log filtering :wink:
Log filtering is way heavier than log rotation.

1 Like

You probably know much more than me, because this is your expertise.

In that case, if logrotate is the simple most widely used log rotation tool, is there an equivalent tool for logs filtering - without requiring a mammoth ELK implementation? (Just to split the access log into separate logs per vhost / service.)

Most comments here and on the repo recommends something like that, but for a small server it's huge.

Sorry to hear you got fired from Traefik Labs, especially as you did development and supported here in the community.

Not sure if it is the right decision for a small tech company to fire their tech people.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.