Traefik proxy 2.10.4 - entrypoint middleware @file dissapears form to the route without errors, but it's present (loaded and configured) in traefik

Traefik Traefik v2
, ,
1

Hello everyone.
First of all I would like to thank to the team for this amazing product (traefik-proxy).

I am using traefik-proxy 2.10.4 and I have run into a very interesting issue, namely with a middleware attached to an entrypoint.
I have been following this advice Configure a middleware for all IngressRoutes and it works, but from time to time one of the middlewares is not applied,(dissapears) to the routes.
I see it in the dashboard/#/http/middlewares dashboard, but when I click it I don't see any ingresses "under it".
I also do not see it loaded into the routes at dashboard/#/http/routers/example-router
There is another middleware, attached to the entrypoint that is a plugin, and it just works fine. The issue is with the built in "headers" middleware.

I have the same setup deployed to 2 other environments, in which everything is working as expected, it's only 1 (at this moment) environment that has this issue.

They are both @file type middlewares.

I suspect this happens when pods get replaced for any reason. In any case it happens after some time has passed and traefik runs for a while.
There is no error in the logs and I see that the middleware is loaded properly, it's just not attached to all the ingress routes.
Any ideas on what can the culprit be, or where can I dig deeper?
Thank you!

2

Share your full config, try to make it reproducible. Where is the difference in your two setups, that one works and one does not?

3

Hi bluepuma77,
thank you for your reply
I will try to sanitize the config first, but there's no difference between the environments.
I just tried and upgraded traefik to 2.10.5 and I immediately got the same issue on the upgraded environment. the remove-headers@file was applied to all the routes and suddenly it wasn't
So now I've 2 envs with the same issue.

I am deploying traefik via helm but testing with docker-compose.
I am looking at the logs as we speak.
Config map with dynamic value.
Here's an excerpt from the config

apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-dynamic-config
data:
  config.yaml: |
    http:
      middlewares:
        x-request-id-header:
          plugin:
            traefik-plugin-request-id:
              Enabled: bool
              Headername: some-header-name
        remove-headers:
          headers:
            customRequestHeaders:
              header-1: ""
              header-2: ""
              header-3: ""

Args passed to traefik

- "--experimental.localplugins.traefik-plugin-request-id.modulename=github.com/mdklapwijk/traefik-plugin-request-id"
- "--providers.file.directory=traefik-dynamic-config"
- "--entrypoints.web.http.middlewares=remove-headers@file"
- "--entrypoints.web.http.middlewares=x-request-id-header@file"
4

Ok I tried something because when I was trialing traefik I noticed that when you name the middlewares in a certain way they don't load, namely I was having issues with the following plugin and name
traefik.http.middlewares.x-request-id.plugin.traefik-plugin-request-id
The issue was so weird that I had to put a comment in the docker-compose file.

## x-request-id-<anything> good
## x-request-id bad

SO what I did is I changed the
from:
- "--entrypoints.web.http.middlewares=remove-headers@file"
to:
- "--entrypoints.web.http.middlewares=removeHeaders@file

And the configmap config file
from:
remove-headers
to:

removeHeaders:
          headers:
            customRequestHeaders:
              header-1: ""
              header-2: ""
              header-3: ""

So far it looks good. Let's see if there's an issue down the line.

It would be nice if the team can have a look and confirm what am I experiencing.

5

If you are convinced its a bug, I would rather open an issue at Traefik Github.

1 Like
6

That's a good idea. I would leave this for a couple of days to see if someone else responds and to see if I don't get the issue anymore.
If both are satisfied I will open an issue on GH.