Inconsistent Connection Handling in Traefik for Multiple Requests: Seeking One Connection Per Request Configuration

Traefik Traefik v2
, ,
1

Hey everyone,

I'm facing an issue where, when making multiple requests in a short period to the web service, Traefik doesn't exhibit a consistent behavior in how it communicates with the service. Sometimes, it opens only one connection, while other times it opens multiple connections. I would like to configure it so that it opens one connection per request.

Scenario:

In a local network, I have a Kali Linux instance generating multiple connections, with Traefik and httpd running in containers. The requests are sent to Traefik, which then forwards them to httpd.

Example:

Requests:
┌──(root㉿kali)-[/home]
└─# python3 pyflooder.py https://site.com.br 4
[#] Attack started on https://site.com.br || # Process: 4
2024-09-06 10:29:23.747174-04:00 --> 0
2024-09-06 10:29:23.750274-04:00 --> 1
2024-09-06 10:29:23.752632-04:00 --> 2
2024-09-06 10:29:23.754832-04:00 --> 3
<Response [200 OK]> --> 3
<Response [200 OK]> --> 1
<Response [200 OK]> --> 0
<Response [200 OK]> --> 2

┌──(root㉿kali)-[/home]
└─# python3 pyflooder.py https://site.com.br 4
[#] Attack started on https://site.com.br || # Process: 4
2024-09-06 10:29:33.438137-04:00 --> 0
2024-09-06 10:29:33.441228-04:00 --> 1
2024-09-06 10:29:33.443563-04:00 --> 2
2024-09-06 10:29:33.445867-04:00 --> 3
<Response [200 OK]> --> 1
<Response [200 OK]> --> 3
<Response [200 OK]> --> 0
<Response [200 OK]> --> 2

┌──(root㉿kali)-[/home]
└─# python3 pyflooder.py https://site.com.br 4
[#] Attack started on https://site.com.br || # Process: 4
2024-09-06 10:29:46.092277-04:00 --> 0
2024-09-06 10:29:46.095272-04:00 --> 1
2024-09-06 10:29:46.097553-04:00 --> 2
2024-09-06 10:29:46.099753-04:00 --> 3
<Response [200 OK]> --> 1
<Response [200 OK]> --> 3
<Response [200 OK]> --> 2
<Response [200 OK]> --> 0

Connection Capture:
root@security82:/usr/local/apache2/conf/extra# tcpdump -n port 443 | grep "[S.]"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:29:23.837626 IP 10.0.2.138.443 > 10.0.2.23.38162: Flags [S.], seq 4228371485, ack 288068240, win 64308, options [mss 1410,sackOK,TS val 4281053236 ecr 163157461,nop,wscale 7], length 0
11:29:23.837775 IP 10.0.2.138.443 > 10.0.2.23.38176: Flags [S.], seq 832929171, ack 1432679390, win 64308, options [mss 1410,sackOK,TS val 4281053236 ecr 163157461,nop,wscale 7], length 0
^C53 packets captured
53 packets received by filter
0 packets dropped by kernel

root@security82:/usr/local/apache2/conf/extra# tcpdump -n port 443 | grep "[S.]"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:29:33.536572 IP 10.0.2.138.443 > 10.0.2.23.45450: Flags [S.], seq 3668057375, ack 73021802, win 64308, options [mss 1410,sackOK,TS val 4281062935 ecr 163167160,nop,wscale 7], length 0
11:29:33.536679 IP 10.0.2.138.443 > 10.0.2.23.45464: Flags [S.], seq 1520219671, ack 3812486819, win 64308, options [mss 1410,sackOK,TS val 4281062935 ecr 163167160,nop,wscale 7], length 0
^C48 packets captured
48 packets received by filter
0 packets dropped by kernel

root@security82:/usr/local/apache2/conf/extra# tcpdump -n port 443 | grep "[S.]"
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:29:46.143465 IP 10.0.2.138.443 > 10.0.2.23.44870: Flags [S.], seq 1735842471, ack 1482066187, win 64308, options [mss 1410,sackOK,TS val 4281075542 ecr 163179767,nop,wscale 7], length 0
11:29:46.147458 IP 10.0.2.138.443 > 10.0.2.23.44876: Flags [S.], seq 2142226614, ack 3538031185, win 64308, options [mss 1410,sackOK,TS val 4281075546 ecr 163179771,nop,wscale 7], length 0
11:29:46.236577 IP 10.0.2.138.443 > 10.0.2.23.44886: Flags [S.], seq 320850004, ack 2686109684, win 64308, options [mss 1410,sackOK,TS val 4281075635 ecr 163179860,nop,wscale 7], length 0
^C68 packets captured
68 packets received by filter
0 packets dropped by kernel

2

Traefik should open <= 1 connection per external request, as it might use an existing internal connection to target service with a http2 connection, which allows multiplexing requests within.

4

Makes sense. Is it possible to make Traefik open a connection per request?

When I make simultaneous requests from two different IPs to the same service, Traefik reuses the connection.

I am working on a solution to mitigate DoS attacks. I need to block IPs according to a request limit.

5

Traefik has middlewares for that (doc)

1 Like
6

I tried using RateLimit but since the connections don't follow a pattern it didn't work in my tests.

Example with default burst 1 and default period 1:

in the file dynamic-traefik.yaml I configured:

http:
  routers:
	rt-apache-websecure:
      entrypoints: websecure
      rule: HostRegexp(`{host:.+}`) && PathPrefix(`/`)
      service: srv-apache-web
      middlewares:
        - ratelimit-asd
  middlewares:
    ratelimit-asd:
      rateLimit:
        average: 3
        #burst: 3
        #period: 1s
  services:
    srv-apache-web:
      loadBalancer:
        servers:
		  - url: '{{env "APACHE_HOSTNAME"}}'

I made 3 requests in the same second

┌──(root㉿kali)-[/home/git/pyflooder]
└─# pyflooder python3.12 pyflooder.py https://example.com 3

2024-09-02 08:19:35.593718-04:00 --> 0
2024-09-02 08:19:35.596678-04:00 --> 1
2024-09-02 08:19:35.598948-04:00 --> 2
<Response [200 OK]> --> 0
<Response [429 Too Many Requests]> --> 1
<Response [200 OK]> --> 2

┌──(root㉿kali)-[/home/git/pyflooder]
└─# pyflooder python3.12 pyflooder.py https://example.com 3

2024-09-02 08:19:44.432737-04:00 --> 0
2024-09-02 08:19:44.435911-04:00 --> 1
2024-09-02 08:19:44.438494-04:00 --> 2
<Response [429 Too Many Requests]> --> 1
<Response [429 Too Many Requests]> --> 2
<Response [200 OK]> --> 0

What I understood from the documentation is that average is divided by period and burst is the maximum volume. Therefore, 3(average) / 1(period) = 3 + 1(burst) = 4 requests within 1 second

Other users reported the same issue with rate-limit:
https://community.traefik.io/t/inconsistency-with-rate-limit-middleware/18190
https://stackoverflow.com/questions/74314530/need-help-understanding-traefik-rate-limit-average-value