We have trying to update Traefik on our development kluster, but after the update both routes and certificates seems to fail.
I have no idea where to look for the issues. If I can supply more information, just let me know!
If anyone can point me in the correct direction I would be very grateful 
Logs from the traefik pod
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-jwgxs namespace=default
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/kubernetes.go:184 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetes
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetescrd
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-ljqnm namespace=traefik-v2
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-jwgxs namespace=default
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/kubernetes.go:184 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetes
2024-12-02T13:01:49Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetescrd
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-ljqnm namespace=traefik-v2
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-jwgxs namespace=default
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetescrd
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/kubernetes.go:184 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetes
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:179 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetescrd
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-ljqnm namespace=traefik-v2
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/client.go:289 > Skipping ingress status update ingress=cm-acme-http-solver-jwgxs namespace=default
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/ingress/kubernetes.go:184 > Skipping Kubernetes event kind *v1.Secret providerName=kubernetes
2024-12-02T13:01:51Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "x0tlb.dev.hololink.io"
2024-12-02T13:01:59Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "auth.dev.hololink.io"
2024-12-02T13:01:59Z DBG log/log.go:245 > http: TLS handshake error from 10.110.0.12:20560: remote error: tls: bad certificate
2024-12-02T13:02:08Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "api.dev.hololink.io"
2024-12-02T13:02:08Z DBG log/log.go:245 > http: TLS handshake error from 10.110.0.12:11464: remote error: tls: unknown certificate
2024-12-02T13:02:10Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "api.dev.hololink.io"
2024-12-02T13:02:10Z DBG log/log.go:245 > http: TLS handshake error from 10.110.0.12:11478: remote error: tls: unknown certificate
2024-12-02T13:02:13Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "x0tlb.dev.hololink.io"
2024-12-02T13:02:15Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "newtest.dev.hololink.io"
2024-12-02T13:02:15Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "x0tlb.dev.hololink.io"
2024-12-02T13:02:18Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: "api.dev.hololink.io"
2024-12-02T13:02:18Z DBG log/log.go:245 > http: TLS handshake error from 10.110.0.12:48904: remote error: tls: bad certificate
dnrn  ~  tmp 
Our Traefik deployment
Name: traefik
Namespace: traefik-v2
CreationTimestamp: Fri, 21 Oct 2022 14:47:12 +0200
Labels: app.kubernetes.io/instance=traefik-traefik-v2
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=traefik
helm.sh/chart=traefik-33.1.0
Annotations: deployment.kubernetes.io/revision: 4
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: traefik-v2
Selector: app.kubernetes.io/instance=traefik-traefik-v2,app.kubernetes.io/name=traefik
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 0 max unavailable, 1 max surge
Pod Template:
Labels: app.kubernetes.io/instance=traefik-traefik-v2
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=traefik
helm.sh/chart=traefik-33.1.0
Annotations: prometheus.io/path: /metrics
prometheus.io/port: 9100
prometheus.io/scrape: true
Service Account: traefik
Containers:
traefik:
Image: docker.io/traefik:v3.2.1
Ports: 9100/TCP, 8080/TCP, 8000/TCP, 8443/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
Args:
--global.checknewversion
--global.sendanonymoususage
--entryPoints.metrics.address=:9100/tcp
--entryPoints.traefik.address=:8080/tcp
--entryPoints.web.address=:8000/tcp
--entryPoints.websecure.address=:8443/tcp
--api.dashboard=true
--ping=true
--metrics.prometheus=true
--metrics.prometheus.entrypoint=metrics
--providers.kubernetescrd
--providers.kubernetescrd.allowEmptyServices=true
--providers.kubernetesingress
--providers.kubernetesingress.allowEmptyServices=true
--providers.kubernetesingress.ingressendpoint.publishedservice=traefik-v2/traefik
--entryPoints.websecure.http.tls=true
--log.level=INFO
--log.level=DEBUG
--entrypoints.websecure.http.tls
--providers.kubernetesIngress.ingressClass=traefik-cert-manager
--ping
--entryPoints.web.proxyProtocol.insecure
--entryPoints.web.forwardedHeaders.insecure
Liveness: http-get http://:8080/ping delay=2s timeout=2s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/ping delay=2s timeout=2s period=10s #success=1 #failure=1
Environment:
POD_NAME: (v1:metadata.name)
POD_NAMESPACE: (v1:metadata.namespace)
Mounts:
/data from data (rw)
/tmp from tmp (rw)
Volumes:
data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
tmp:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Node-Selectors: <none>
Tolerations: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: traefik-6d7bd8d6bc (0/0 replicas created), traefik-6bf844d485 (0/0 replicas created), traefik-6f77ccb68c (0/0 replicas created)
NewReplicaSet: traefik-76cf75db45 (1/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 57m deployment-controller Scaled up replica set traefik-6f77ccb68c to 1
Normal ScalingReplicaSet 56m deployment-controller Scaled down replica set traefik-6bf844d485 to 0 from 1
Normal ScalingReplicaSet 34m deployment-controller Scaled up replica set traefik-76cf75db45 to 1
Normal ScalingReplicaSet 34m deployment-controller Scaled down replica set traefik-6f77ccb68c to 0 from 1