Every time our Traefik instance reaches a higher number (1k>) of connections, it starts to return non-TLS content on the HTTPS endpoint. Body contains:
HTTP/1.1 400 Bad Request Content-Type: text/plain; charset=utf-8 Connection: close 400 Bad Request
Memory and CPU usage are both under 10% of usage.
- Traefik docker image version: docker.io/traefik:v2.10.1
- Helm chart version: v23.1.0
I tried versions 20.8.0, 21.2.1, 22.0.0, and 22.3.0 charts with the same result.
To be sure there are no cross-wires, I disabled metrics and web endpoint, leaving only websecure entrypoint. The result was the same.
The issue happens regardless of the number of replicas.
Our setup is running as NodePort behind AWS NLB.
We use arm instances, but the issue also happens for the amd64 architecture.
Believe it or not, nginx has no issue like this running in the same cloud, same setup, same traffic.
Has anyone also experienced this issue? Thx