can't access to dashboard

Traefik Traefik v2 (latest)
1

What version of the Traefik's Helm Chart are you using?

3.5.0

What version of Traefik are you using?

2.6.6

What did you do?

Trying to access the dashboard.

I have deployment.yaml

---
# Source: traefik/templates/deployment.yaml
# Source: traefik/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-traefik-internal-controller
  labels:
    app.kubernetes.io/name: traefik
    helm.sh/chart: traefik-10.19.5
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/instance: ingress-traefik-internal-controller
  annotations:
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik
      app.kubernetes.io/instance: ingress-traefik-internal-controller
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  minReadySeconds: 0
  template: 
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: "/metrics"
        prometheus.io/port: "9100"
      labels:
        app.kubernetes.io/name: traefik
        helm.sh/chart: traefik-10.19.5
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/instance: ingress-traefik-internal-controller
    spec:
      serviceAccountName: ingress-traefik-internal-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: false
      containers:
      - image: "traefik:2.6.6"
        imagePullPolicy: IfNotPresent
        name: ingress-traefik-internal-controller
        resources:
        readinessProbe:
          httpGet:
            path: /ping
            port: 9000
          failureThreshold: 1
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
        livenessProbe:
          httpGet:
            path: /ping
            port: 9000
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
        ports:
        - name: "metrics"
          containerPort: 9100
          protocol: "TCP"
        - name: "traefik"
          containerPort: 9000
          protocol: "TCP"
        - name: "web"
          containerPort: 8000
          protocol: "TCP"
        - name: "websecure"
          containerPort: 8443
          protocol: "TCP"
        securityContext:
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsGroup: 65532
          runAsNonRoot: true
          runAsUser: 65532
        volumeMounts:
          - name: data
            mountPath: /data
          - name: tmp
            mountPath: /tmp
        args:
          - "--global.checknewversion"
          - "--global.sendanonymoususage"
          - "--entrypoints.metrics.address=:9100/tcp"
          - "--entrypoints.traefik.address=:9000/tcp"
          - "--entrypoints.web.address=:8000/tcp"
          - "--entrypoints.websecure.address=:8443/tcp"
          - "--api.dashboard=true"
          - "--ping=true"
          - "--metrics.prometheus=true"
          - "--metrics.prometheus.entrypoint=metrics"
          - "--providers.kubernetescrd"
          - "--providers.kubernetesingress"
          - "--entrypoints.websecure.http.tls=true"
          - "--log.level=INFO"
          - "--accesslog=true"
          - "--accesslog.fields.defaultmode=keep"
          - "--accesslog.fields.headers.defaultmode=drop"
      volumes:
        - name: data
          emptyDir: {}
        - name: tmp
          emptyDir: {}
      securityContext:
        fsGroup: 65532

IngressRoute:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: ingress-traefik-internal-controller-dashboard
  namespace: "traefik-internal"
  labels:
    app.kubernetes.io/name: ingress-traefik-internal-controller-dashboard
    helm.sh/chart: ingress-traefik-internal-controller-dashboard-1.0
    app.kubernetes.io/instance: ingress-traefik-internal-controller
    app.kubernetes.io/managed-by: Helm
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: PathPrefix(`/traefik`)
      services:
      - kind: TraefikService
        name: api@internal

However, when I tried to access /traefik, traefik printed out this log

10.2.1.1 - - [28/Jun/2022:06:43:06 +0000] "GET /traefik HTTP/1.1" 404 19 "-" "-" 20 "traefik-ingress-traefik-controller-dashboard-27f2a6f95cea93d2ebaf@kubernetescrd" "-" 0ms

routing is doing its job but traefik-ingress-traefik-controller-dashboard-27f2a6f95cea93d2ebaf@kubernetescrd is returning 404?

How can I debug this?