Service koolaidevents is deployed separately via Gitlab CI/CD and it detects it automatically since they are in the same namespace. I also have a different service in this same cluster with its own set of pods. The problem is it's load-balancing the requests across all pods in the cluster. I expect it to to all load-balance across pods only within koolaidevents. Whenever the request hits any of the pods in the other service, it just returns a 502 - Bad Gateway.
I'm totally new to using Traefik. Please help out.
Purely anecdotal: I saw a cluster with a bunch of traefik ingresses and one of the ingresses by mistake had a wild card host rule. Naturally all the other apps from time to time would hit that ingress, since it would match anything, which resulted in a wrong app being served.
The takeaway is: other apps ingresses can affect routing if your app.
@zespri Thanks a lot for your time. I really appreciate. I removed the other service while I was confused and trying to debug. I have updated the above configuration below to include the other service.
The issue is actually in the application. It's listening on all interfaces 0.0.0.0
I changed that to localhost still didn't work and had to use Node.js os and dns libraries to resolve the Pod's IP address and it worked.
I'm still not totally clear though. I thought just specifying the service name as I did above should correctly route all requests that match that host to the specific service's ClusterIP which should handle the routing to the appropriate pod. Am I missing something ?
By and large. There is an implementation detail, that for performance reasons traefik does not use service's ClusterIP, but uses the endpoints directly. The result though must be the same. It is though important that those endpoints are what you expect them to be.
Sorry, I have no idea what you are talking about. What application? What "didn't work"? What pod's IP address? How is it used in the application?
It's a Node.js server (fastify to be precise) that was listening on 0.0.0.0 which listens on all interfaces. fastify.listen(PORT, '0.0.0.0')
What I did to resolve the issue was get the Kubernetes pod's IP with dns.resolve(os.hostname(), (err, addr) => ...)
then listen on the addr returned.
Now the issue has resurfaced after I deployed like 5 frontends. Trying to access one of the frontend and Traefik started routing to all the pods and I can confirm that each of these pods is listening on its Pod's IP and a different port.
The routing is random. I have 3 pods per each service and I expected that only these 3 pods should receive the requests since the Rule says so. This is not so and pods in other services matched by a different Rule also get the request sometimes at random.