Getting nothing but 404 on Dashboard, even when port-forwarding straight to port on pod

Hi, running Traefik 2.8.1 (after migrating from 1.x) and I'm unable to see Traefik Dashboard. I thought my IngressRoute setup was wrong so I tried to port-forward directly to port 8080 on the dashboard service but I get a 404. I even tried to port-forward straight to port 8080 on a Traefik pod. Every time I get a 404!

Why is dashboard not responding at all?

All resources run in the traefik namespace.

---
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: traefik
  labels:
    app: traefik-ingress-controller
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  selector:
    matchLabels:
      app: traefik-ingress-controller
  template:
    metadata:
      labels:
        app: traefik-ingress-controller
        name: traefik-ingress-controller
    spec:
      priorityClassName: insiders-high
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
        - name: traefik-ingress-configmap
          configMap:
            name: traefik-ingress-configmap
      containers:
      - image: traefik:v2.8
        name: traefik-ingress-controller
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: ${CPU_REQUESTS}
            memory: ${MEM_REQUESTS}
          limits:
            memory: ${MEM_LIMITS}
        livenessProbe:
          failureThreshold: 2
          httpGet:
            path: /ping
            port: 80
            scheme: HTTP
          periodSeconds: 5
        readinessProbe:
          failureThreshold: 2
          httpGet:
            path: /ping
            port: 80
            scheme: HTTP
          periodSeconds: 5
        volumeMounts:
          - mountPath: "/etc/traefik"
            name: "traefik-ingress-configmap"
        ports:
        - name: web
          containerPort: 80
        - name: websecure
          containerPort: 443
        - name: dashboard
          containerPort: 8080
        - name: metrics
          containerPort: 9090
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-ingress-configmap
  namespace: traefik
data:
  traefik.yml: |
    global:
      sendAnonymousUsage: true
    log:
      level: DEBUG
      format: json
    accessLog:
      format: json
    providers:
      kubernetesIngress:
        ingressClass: "traefik"
        ingressEndpoint:
          publishedService: "traefik/traefik-ingress-service"
      kubernetesCRD:
        ingressClass: "traefik"
    entrypoints:
      web:
        address: ":80"
        forwardedHeaders:
          insecure: true
        http:
          redirections:
            entrypoint:
              to: websecure
              scheme: https
      websecure:
        address: ":443"
        forwardedHeaders:
          insecure: true
        http:
          tls: {}
      metrics:
        address: ":9090"
      traefik:
        address: ":8080"
    ping:
      entrypoint: "web"
    dashboard:
      entrypoint: "traefik"
    metrics:
      prometheus:
        buckets:
          - 0.1
          - 0.3
          - 1.2
          - 5
        entrypoint: metrics
    api:
      dashboard: true
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-dashboard
  namespace: traefik
  labels:
    app: traefik-dashboard
spec:
  selector:
    app: traefik-ingress-controller
  ports:
  - port: 8080
    name: dashboard

When opening a port-forwarded dashboard the logs on a pod show:

time="2022-07-26T16:02:41Z" level=info msg="Configuration loaded from file: /etc/traefik/traefik.yml"
{"level":"info","msg":"Traefik version 2.8.1 built on 2022-07-11T14:06:57Z","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Stats collection is enabled.","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Many thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration.","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Help us improve Traefik by leaving this feature on :)","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"More details on: https://doc.traefik.io/traefik/contributing/data-collection/","time":"2022-07-26T16:02:41Z"}
{"level":"warning","msg":"Traefik Pilot is deprecated and will be removed soon. Please check our Blog for migration instructions later this year.","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Starting provider aggregator aggregator.ProviderAggregator","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Starting provider *traefik.Provider","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Starting provider *ingress.Provider","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"ingress label selector is: \"\"","providerName":"kubernetes","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Starting provider *crd.Provider","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Creating in-cluster Provider client","providerName":"kubernetes","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Starting provider *acme.ChallengeTLSALPN","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"label selector is: \"\"","providerName":"kubernetescrd","time":"2022-07-26T16:02:41Z"}
{"level":"info","msg":"Creating in-cluster Provider client","providerName":"kubernetescrd","time":"2022-07-26T16:02:41Z"}
{"ClientAddr":"127.0.0.1:39696","ClientHost":"127.0.0.1","ClientPort":"39696","ClientUsername":"-","DownstreamContentSize":19,"DownstreamStatus":404,"Duration":83015,"Overhead":83015,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":74,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"StartLocal":"2022-07-26T16:04:14.546327452Z","StartUTC":"2022-07-26T16:04:14.546327452Z","level":"info","msg":"","time":"2022-07-26T16:04:14Z"}
{"ClientAddr":"127.0.0.1:39696","ClientHost":"127.0.0.1","ClientPort":"39696","ClientUsername":"-","DownstreamContentSize":19,"DownstreamStatus":404,"Duration":50359,"Overhead":50359,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":75,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/favicon.ico","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"StartLocal":"2022-07-26T16:04:14.639176779Z","StartUTC":"2022-07-26T16:04:14.639176779Z","level":"info","msg":"","time":"2022-07-26T16:04:14Z"}
{"ClientAddr":"127.0.0.1:39760","ClientHost":"127.0.0.1","ClientPort":"39760","ClientUsername":"-","DownstreamContentSize":19,"DownstreamStatus":404,"Duration":118110,"Overhead":118110,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":78,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"StartLocal":"2022-07-26T16:04:17.444746888Z","StartUTC":"2022-07-26T16:04:17.444746888Z","level":"info","msg":"","time":"2022-07-26T16:04:17Z"}
{"ClientAddr":"127.0.0.1:39760","ClientHost":"127.0.0.1","ClientPort":"39760","ClientUsername":"-","DownstreamContentSize":19,"DownstreamStatus":404,"Duration":51226,"Overhead":51226,"RequestAddr":"localhost:8080","RequestContentSize":0,"RequestCount":83,"RequestHost":"localhost","RequestMethod":"GET","RequestPath":"/dashboard/","RequestPort":"8080","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"StartLocal":"2022-07-26T16:04:21.666094217Z","StartUTC":"2022-07-26T16:04:21.666094217Z","level":"info","msg":"","time":"2022-07-26T16:04:21Z"}

Well, I figured it out myself.

The reason I could't see anything even when port-forwarding to a pod was because the dashboard was forbidden from accepting insecure connections. I would've prefered to get a 403 response instead to point me in the direction of my error more quickly, but at least I now understand why it wasn't working.

I was also unable to access the dashboard through a ingressRoute with or without TLS. Turns out my mistake here was that I'd attached the IngressRoute to the traefik entryPoint, while I should've attached it to the websecure entryPoint through which my TLS encrypted external traffic was coming. Changing the entrypoint made the dashboard work right away even with insecure set to false (since the traffic is now coming through a secure connection)

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.