Traefik's throughput suddenly ramps up slower after running for a while

I run Traefik and another container in Docker Swarm both using Docker overlay networks. The other container serves larger payloads (think of streaming data or large static files).

When I start Traefik to serve as a reverse proxy, it performs very well. But after running for some time (non-determinate, usually a few hours), when making new requests, the throughput from Traefik ramps up much slower than bypassing Traefik.

I measured this using curl and captured pcaps, and plotted the throughput as shown in the attachement. The first row shows Traefik just starting, second row was after a while (this time is about 1 hour). The first & second column show throughput via Traefik, the thrid column is bypassing traefik.

I can't find any relevant information and would like to know if anyone else has seen anything similar.