Traefik can't connect to Kubernetes service (504 Gateway timeout)

Traefik Traefik v3 (latest)
1

I’m having trouble connecting to my backend service via the Traefik ingress class.

I have a backend pod with service running. I’ve connected the Ingress resource to it

k get pods,svc -n app
NAME                                      READY   STATUS    RESTARTS   AGE
pod/app-ria-64cc85c68b-l7244      1/1     Running   0          14m

NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/app-ria    ClusterIP   172.26.81.232   <none>        8080/TCP   13d

With the following ingress spec

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ria
  namespace: app
spec:
  ingressClassName: traefik
  rules:
  - host: app
    http:
      paths:
      - backend:
          service:
            name: app-ria
            port:
              number: 8080
        path: /
        pathType: Prefix
  - host: app.subdomain.mycompany.tld
    http:
      paths:
      - backend:
          service:
            name: app-ria
            port:
              number: 8080
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - app
    - app.subdomain.mycompany.tld
    secretName: ingress-secret

Connecting to one of my hostnames yields a 504 (on every path). With below debug log:

│
│ 2026-04-08T11:24:56Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:288 > Serving default certificate for request: ""                                                                                                            │
│ 2026-04-08T11:24:56Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:251 > Service selected by WRR: http://172.25.4.205:8080                                                                                 │
│ 2026-04-08T11:25:02Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:130 > Skipping Kubernetes event kind *v1.ConfigMap providerName=kubernetescrd                                                            │
│ 2026-04-08T11:25:17Z DBG github.com/traefik/traefik/v3/pkg/provider/kubernetes/crd/kubernetes.go:130 > Skipping Kubernetes event kind *v1.ConfigMap providerName=kubernetescrd                                                            │
│ 2026-04-08T11:25:26Z DBG github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:121 > 504 Gateway Timeout error="dial tcp 172.25.4.205:8080: i/o timeout"                                                                              │
│ 10.42.229.201 - - [08/Apr/2026:11:24:56 +0000] "GET / HTTP/1.1" 504 15 "-" "-" 9 "websecure-app-app-ria-app.subdomain.mycompany.tld@kubernetes" "http://172.25.4.205:8080" 30001ms

I’m not sure what’s going wrong here. Port-forwarding my app’s svc to a local port works just fine, using NGINX as the ingress class also works fine.

I’ve already tried to use Traefik with the NGINX annotations (providers.kubernetesIngressNginx) but to no avail.

These are the NGINX annotations we use in production

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/affinity-canary-behavior: sticky
    nginx.ingress.kubernetes.io/affinity-mode: persistent
    nginx.ingress.kubernetes.io/app-root: /path/
    nginx.ingress.kubernetes.io/proxy-body-size: 100m
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "6000"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "6000"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "6000"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/session-cookie-max-age: "43200"
    nginx.ingress.kubernetes.io/session-cookie-secure: "true"

Does anyone know if Traefik modifies my request to the backend by default?