K8s-ingress is not working with error like msg="Cannot create service: service port not found"

Traefik Traefik v2 (latest)
1

I am trying to make something simple and I guess I am missing something :slight_smile:

I have a pod running docker-registry , listening port is 5000

          value: '0.0.0.0:5000'

then the service is redirecting port 5001 to pod port 5000

apiVersion: v1
kind: Service
metadata:
  name: docker-registry
  namespace: docker-registry
  labels:
    k3s-app: docker-registry
spec:
  selector:
    k3s-app: docker-registry
  type: ClusterIP
  ports:
  - name: http
    port: 5001
    targetPort: 5000

finally I have an ingress targeting port 5001

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: registry-ingress
  namespace: docker-registry
spec:
  rules:
  - host: registry
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: docker-registry 
            port:
              number: 5001

registry is a round robing DNS to all IPs of the nodes of my cluster (172.16.1.210-214)

kubectl get svc -n docker-registry
NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
docker-registry   ClusterIP   10.43.117.62   <none>        5001/TCP   23h

kubectl get ingress -n docker-registry
NAME               CLASS    HOSTS                            ADDRESS                                                            PORTS     AGE
registry-ingress   <none>   registry   172.16.1.210,172.16.1.211,172.16.1.212,172.16.1.213,172.16.1.214   80, 443   23h

when I use 10.43.117.62:5001 for the registry, it is working, authentication is ok, I have good logs from the pod
when I use registry or any IP of the node, I have a status 400, and no log on the pod.

what did I miss ?
I have exactly the same type of ingress working with longhorn and dashboard pods, but the URL is something like http://longhorn/#/backup or http://longhorn/#/nodes, with the registry, thr URL is more like http[s]://registry/v2/ or http[s]://registry/v1/_ping ...

thanks for your help
Fox