I've had... A journey today... I went to move my homelab Traefik install (Containerd/Kuberentes) from HTTP challenge to DNS for TLS... When that didn't appear to be working (certs didn't generate), I went to upgrade my Traefik container image from 2.7 to 3.3 thinking something in there might fix it.
For some reason, updating the image broke every router and they dropped from the Traefik dashboard.
I ended up deleting all of the roles, CRDs, everything Traefik and doing something of a fresh install using the links in the tutorial.
I have the routers working again (and they seem snappier), but now I am getting a flood of errors like these in the logs:
W0311 05:13:47.999053 1 reflector.go:561] k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: failed to list *v1.GRPCRoute: grpcroutes.gateway.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "grpcroutes" in API group "gateway.networking.k8s.io" at the cluster scope
E0311 05:13:47.999130 1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: Failed to watch *v1.GRPCRoute: failed to list *v1.GRPCRoute: grpcroutes.gateway.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "grpcroutes" in API group "gateway.networking.k8s.io" at the cluster scope" logger="UnhandledError"
W0311 05:13:56.624419 1 reflector.go:561] k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: failed to list *v1alpha3.BackendTLSPolicy: backendtlspolicies.gateway.networking.k8s.io is forbidden: User "system:serviceaccount:default:traefik-ingress-controller" cannot list resource "backendtlspolicies" in API group "gateway.networking.k8s.io" at the cluster scope
But the thing is, traefik-ingress-controller doesn't have tlsroutes or gateway.networking.k8s.io in the ClusterRole for traefik-ingress-controller. And nothing that I have deployed uses these fields... So where the heck did this come from?
Any ideas?
My Kubernetes deployment for reference:
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: default
name: traefik-ingress-controller
kind: Deployment
apiVersion: apps/v1
metadata:
namespace: default
name: traefik
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v3.3
args:
- --log.level=warn
- --api.insecure
- --accesslog
- --api.dashboard=true
- --ping=true
- --entrypoints.web.Address=:80
- --entryPoints.web.forwardedHeaders.insecure
- --entryPoints.web.proxyProtocol.insecure
- --entrypoints.websecure.Address=:443
- --entryPoints.websecure.forwardedHeaders.insecure
- --entryPoints.websecure.proxyProtocol.insecure
- --entryPoints.metrics.address=:8082
- --entryPoints.metrics.forwardedHeaders.insecure
- --entryPoints.plex.Address=:32400
- --entryPoints.plex.forwardedHeaders.insecure
- --entryPoints.minecraft.Address=:25565
- --entryPoints.minecraft.forwardedHeaders.insecure
- --certificatesresolvers.myresolver.acme.dnschallenge=true
- --certificatesresolvers.myresolver.acme.dnschallenge.provider=cloudflare
- --certificatesresolvers.myresolver.acme.email=myemail@gmail.com
- --certificatesresolvers.myresolver.acme.storage=/certs/acme.json
#- --certificatesresolvers.le.acme.httpchallenge=true
#- --certificatesresolvers.le.acme.httpchallenge.entrypoint=http
#- --certificatesresolvers.le.acme.tlschallenge=true
#- --providers.kubernetescrd.endpoint=http://localhost:8080
- --providers.kubernetesgateway.experimentalchannel=true
- --providers.kubernetescrd=true
- --providers.kubernetesIngress=true
#- --metrics.prometheus=true
#- --metrics.prometheus.buckets=0.100000, 0.300000, 1.200000, 5.000000
#- --metrics.prometheus.addEntryPointsLabels=true
#- --metrics.prometheus.addServicesLabels=true
#- --metrics.prometheus.entryPoint=metrics
#- --pilot.token=556978c7-e84c-4407-8631-da3c32c3a800
env:
- name: CF_DNS_API_TOKEN
value: "q9nasdVb_Yqi6SXsH2kn0"
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: admin
containerPort: 8080
- name: metrics
containerPort: 8082
- name: plex
containerPort: 32400
- name: minecraft
containerPort: 25565
- name: torrent
containerPort: 8999
volumeMounts:
- name: traefik-cert
mountPath: "/certs"
volumes:
- name: traefik-cert
hostPath:
path: /mnt/nvme/traefik
kind: Service
apiVersion: v1
metadata:
name: traefik
namespace: default
spec:
selector:
app: traefik
ports:
- protocol: TCP
name: http
port: 80
nodePort: 30180
- protocol: TCP
name: https
port: 443
nodePort: 30143
- protocol: TCP
name: admin
port: 8080
nodePort: 30108
- protocol: TCP
name: plex
port: 32400
nodePort: 30124
- protocol: TCP
name: metrics
port: 8082
nodePort: 30102
- protocol: TCP
name: minecraft
port: 25565
nodePort: 30155
type: NodePort
Use 3 backticks before and after code/config to preserve spacing, which is important im yaml.
You know best what you use, I can not comment on your config.
I suggest reading the v2 to v3 migration guide, traefik has good migration documentation, it helped me alot: