502 errors creating an ingressroute for kube-prometheus grafana

Traefik Traefik v2
,
1

I am using traefik as part of my k3s ( 1.24.4+k3s1) cluster, along with kube-prometheus (v0.11) to monitor my infrastructure.

I've decided to expose my grafana depoloyment via an IngressRoute. I already had some existing ingressroutes which worked, so i simply adapted those to work with grafana. However, for some reason, this doesnt seem to work - i get 502s all over.

When I set the traefik logs to debug, I see lots of '502 Bad Gateway' caused by: dial tcp 10.42.0.16:3000: connect: connection refused errors whenever i try to access my service. Indeed, if i exec into that container and do a wget, that address refuses to connect.

However, if i exec into another container, the wget works fine.

I dont see how this makes any sense - does anyone know what the issue might be?

2

Update to future travellers:

I figured it out - kube-prometheus added a NetworkPolicy which was blocking my ingress. Updating the policy fixed the ingress. All good!