Why secret for basic auth generate Error log?

[obeyler]

Middleware auth creates bad logs : Even with the secret I always have the log

level=error msg="Error while reading basic auth middleware: failed to load auth credentials: secret 'traefik/traefik' not found" middlewareName=traefik-auth providerName=kubernetescrd
level=error msg="middleware \"traefik-auth@kubernetescrd\" does not exist" routerName=traefik-traefik-86717a050e7497cf6af6@kubernetescrd entryPointName=traefik
level=error msg="middleware \"traefik-auth@kubernetescrd\" does not exist" entryPointName=web routerName=traefik-traefik-86717a050e7497cf6af6@kubernetescrd
level=error msg="Error while reading basic auth middleware: failed to load auth credentials: secret 'traefik/traefik' not found" providerName=kubernetescrd middlewareName=traefik-auth

see full log at the end of this issue

What did you do?

this is my Middleware:
kubectl get middleware -n traefik auth -o yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: auth
  namespace: traefik
spec:
  basicAuth:
    secret: traefik

this is my secret

kubectl get secret -n traefik traefik -o yaml

apiVersion: v1
data:
  users: YWRtaW46JGFwcjEkS1JhemRoMkokcDN2NW03aWJ6MTJLMGlFeGIuZ05NMA==
kind: Secret
metadata:   
  name: traefik
  namespace: traefik
type: Opaque

this is my ingressroute
kubectl get ingressroute -n traefik traefik -o yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik
  namespace: traefik
spec:
  routes:
  - kind: Rule
    match: Host(`traefik-ui.k8s-serv.nd-int-ops-paas.itn`) && (PathPrefix(`/api`)
      ||PathPrefix(`/dashboard`))
    middlewares:
    - name: auth
      namespace: traefik
    priority: 1
    services:
    - kind: TraefikService

What did you expect to see?

no log error

What did you see instead?

log error
and IngressRoute doesn't works

Output of traefik version: (What version of Traefik are you using?)

traefik:v2.1

What is your environment & configuration (arguments, toml, provider, platform, ...)?

      - args:
        - --api=true
        - --api.dashboard=true
        - --global.sendanonymoususage=true
        - --log.level=INFO
        - --providers.kubernetescrd=true
        - --entrypoints.web=true
        - --entryPoints.web.address=:80
        - --metrics.prometheus=true

If applicable, please paste the log output in DEBUG level (--log.level=DEBUG switch)

kubectl logs -n traefik traefik-ingress-controller-sczr7 
time="2020-02-21T09:20:43Z" level=info msg="Configuration loaded from flags."
time="2020-02-21T09:20:43Z" level=info msg="Traefik version 2.1.4 built on 2020-02-06T17:10:06Z"
time="2020-02-21T09:20:43Z" level=info msg="Stats collection is enabled."
time="2020-02-21T09:20:43Z" level=info msg="Many thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration."
time="2020-02-21T09:20:43Z" level=info msg="Help us improve Traefik by leaving this feature on :)"
time="2020-02-21T09:20:43Z" level=info msg="More details on: https://docs.traefik.io/v2.0/contributing/data-collection/"
time="2020-02-21T09:20:43Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
time="2020-02-21T09:20:43Z" level=info msg="Starting provider *traefik.Provider {}"
time="2020-02-21T09:20:43Z" level=info msg="Starting provider *crd.Provider {}"
time="2020-02-21T09:20:43Z" level=info msg="label selector is: \"\"" providerName=kubernetescrd
time="2020-02-21T09:20:43Z" level=info msg="Creating in-cluster Provider client" providerName=kubernetescrd
time="2020-02-21T09:20:43Z" level=error msg="Error while reading basic auth middleware: failed to load auth credentials: secret 'traefik/traefik' not found" middlewareName=traefik-auth providerName=kubernetescrd
time="2020-02-21T09:20:43Z" level=error msg="middleware \"traefik-auth@kubernetescrd\" does not exist" routerName=traefik-traefik-86717a050e7497cf6af6@kubernetescrd entryPointName=traefik
time="2020-02-21T09:20:43Z" level=error msg="middleware \"traefik-auth@kubernetescrd\" does not exist" entryPointName=web routerName=traefik-traefik-86717a050e7497cf6af6@kubernetescrd
time="2020-02-21T09:20:43Z" level=error msg="Error while reading basic auth middleware: failed to load auth credentials: secret 'traefik/traefik' not found" providerName=kubernetescrd middlewareName=traefik-auth

[ldez]

Hello,

@obeyler

secret 'traefik/traefik' not found

this error come when the official k8s client cannot found the secret

• traefik/pkg/provider/kubernetes/crd/kubernetes.go at bb4de11c517dfa4a6f6ca446732f4b55f771cb49 · traefik/traefik · GitHub
• traefik/pkg/provider/kubernetes/crd/client.go at bb4de11c517dfa4a6f6ca446732f4b55f771cb49 · traefik/traefik · GitHub

so it's seem to not be an issue with Traefik but with your cluster configuration.

[obeyler]

@ldez,
May it's due to the fact that this K8S cluster config is the reason of this log. It is installed by bosh. It's a CFCR cluster (a K8S certified Kubernetes distribution it's the open source part of PKS ) . The path of some configuration files are not same as in kubeadm installation.
Whatever, the K8S cluster works fine with lot of product installed on it (Gitlab, Minios,KubeAdmin, ldap, metabase, postgres, cassendra,... about a 30products run well on it, even Traefik 1.72. works well) So I have some doubt about a bad configuration of this cluster.

You said The official client not found it. but may be the official client use inside the traefik v2 can be bad configured or it just don't be able to get his config.
As you can also see the offical client outside the traefik is able to find the secret
kubectl get secret -n traefik traefik -o yaml

apiVersion: v1
data:
  users: YWRtaW46JGFwcjEkS1JhemRoMkokcDN2NW03aWJ6MTJLMGlFeGIuZ05NMA==
kind: Secret
metadata:   
  name: traefik
  namespace: traefik
type: Opaque

Note also that the traefik V2 is also able to find this secret by another way as the basic auth is applied to access to the ui of the traefik.

The other effect of this log is that all other IngressRoute are not working. Just Ingress works on Traefik V2 and the IngressRoute use by the Traefik ui (by api way+ basic auth)

[obeyler]

I exchanged with Michael (Thanks a lot !) we have seen that the error log of missing secret is a real pb (these log entry should not exist as the secret really exist and it is use by the middleware). It may be due to the fact that the configuration file of K8S is not located as usual path. (need more investigation)
But It has no impact on the fact that the IngressRoutes didn't work.
With Mickael we have changed the config and make the IngressRoutes work well.
Thanks a lot again nice job.

[juranir]

It's a old thread, but I'm the same problem , is possible you share how you solved this issue?