We have been using Treafik without issues until we upgraded kubernetes from 1.21 to 1.23.
Everything seemed to work fine until we had to create a new SSL certificate with cert-manager. The SSL challenges failed because of a 404 error. After several hours we suspected the issue could be related to our version of Treafik and decided to upgrade from 2.4.8 to 2.5.6.
The upgrade was done with helm with the following addition values:
additionalArguments:
- "--log.level=DEBUG"
- "--entrypoints.websecure.http.tls"
- "--ping"
- "--metrics.prometheus"
helm upgrade -install traefik traefik/traefik --version v10.9.1 --namespace traefik -f ops/kubernetes/traefik/values.yaml
All the challenges started to get resolved and the hopes where high.
But we went from bad to worse.
All IngressRoutes seems to have stoped working and all endpoint returned 404.
Ex our API ingress route looks like this and stoped working after the upgrade:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: api-ingress-route
spec:
entryPoints:
- websecure
routes:
- match: Host(`api.hololink.io`)
kind: Rule
services:
- name: api-service
port: 80
tls:
secretName: api-cert
We have no clue what's going on, and got some serious troubles at the moment, where we are not able to deploy new apps to our cluster..