I'm in process to extend my traefik installation with few applications with crowdsec. I added crowdsec container, consumed the logs, added the bouncer - so far everything works.
After few weeks crowdsec console told me there are too many alerts for a free tier and I should switch to enterprise subscription - which is an absolute overkill for a private instance with one family as users.. so I checked the logs and found many requests crowdsec counts as "alerts" are very dumb crawler which don't even try to access a service but simply connect to my public IP and enumerate some common URIs like /.git
,/.env
and similar so I decided to create a "catchall" router (credits to https://stackoverflow.com/questions/60321335/default-router-with-traefik)
router
routers:
catchall-router:
# PathPrefix(/) will match any request because all paths start with /
rule: "PathPrefix(`/`)"
service: catchall@file
# lower number - lowest priority
priority: -999
middlewares:
- deny-all-ip@file
tls: {}
entryPoints:
- web-secure
- take all requests not handled by other routers
- because of PathPrefix / (basicaly speaking everything)
- and low priority -999
- process the request through "deny-all-ip" middleware - handling all weird requests with 403
this covered most of the crawlers but still there are some leftovers like this
request
{
"ClientAddr": "164.52.24.188:38707",
"ClientHost": "164.52.24.188",
"ClientPort": "38707",
"ClientUsername": "-",
"DownstreamContentSize": 19,
"DownstreamStatus": 404,
"Duration": 143702,
"GzipRatio": 0,
"OriginContentSize": 0,
"OriginDuration": 0,
"OriginStatus": 0,
"Overhead": 143702,
"RequestAddr": "83.xxx.xxx.86",
"RequestContentSize": 0,
"RequestCount": 324296,
"RequestHost": "83.xxx.xxx.86",
"RequestMethod": "GET",
"RequestPath": "/favicon.ico",
"RequestPort": "-",
"RequestProtocol": "HTTP/1.1",
"RequestScheme": "https",
"RetryAttempts": 0,
"SpanId": "0000000000000000",
"StartLocal": "2025-05-21T21:23:28.591126147+02:00",
"StartUTC": "2025-05-21T19:23:28.591126147Z",
"TLSCipher": "TLS_AES_128_GCM_SHA256",
"TLSVersion": "1.3",
"TraceId": "00000000000000000000000000000000",
"downstream_Content-Type": "text/plain; charset=utf-8",
"entryPointName": "web",
"level": "info",
"msg": "",
"request_User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36",
"request_X-Real-Ip": "164.52.24.188",
"time": "2025-05-21T21:23:28+02:00"
}
which made me wonder - why this request doesn't hit the catchall
router like all other requests simply connecting to the IP.
It took me a while until I spot the request hits plain-http entryPoint "web"
but uses "RequestScheme": "https"
.
Testing with curl.exe -k -v https://83.xxx.xxx.86:80/
confirms the issue (notice port :80
in the URI):
my web entryPoint redirect to web-secure
entryPoint redirect
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint:
to: web-secure
scheme: https
permanent: true
doesn't work for requests using https scheme.
I solved my specific problem adding web
entryPoint to my router but still wondering if this is expected or rather a bug why the request to plain-http entrypoint using https scheme doesn't follow the redirections directive?