With that in mind, I see how this really is a different problem that is indeed unrelated to the original observation. Well, mostly unrelated: The case "healthcheck enabled AND healthy", while not 404ing, still prints the missing middleware errors. This, I believe, has the same root cause as the original issue?
Right, I think this is the same root cause. Without a container health check, the error message could be printed even if everything is working as expected. As said in a previous message, this might be because the Docker API is a bit flaky and does not return all containers, but things should converge at the end, and routers/services should be reachable.
Does it answer all your questions?