Blocking curl and similar cmds with traefik

Traefik Traefik v3 (latest)
1

I have a url that looks as:

https://www.mysite_sample.net/api

I want to block curl and any other similar program from reaching this link i.e. if this link is used by curl or similar then the response will be an "error" one.

However, I need the sub link

https://www.mysite_sample.net/api/help/index.html

to be available for my site.

How can this be done with Traefik in Kubernetes?

2

Identifying the browser/client is done by the "agent", which is a string sent by the client with every request. But this means that it can be faked. So there is no secure method to identify the client requesting a URL.

3

I was thinking on something as:


- match: Host(`www.mysite_sample.net`) && PathRegexp(`/api/^(?:(?!.*\help\b).)*$`)
  kind: Rule
  services:
    - kind: Service
      name: headless-service-4-null
      port: 8080
      namespace: kube-system

but I guess my REGex is not working. headless-service-4-null is just a null service that sends back a non existing server message.

4

So you want to block certain paths, not certain clients like curl?

5

If any traffic to www.mysite_sample.net/api is blocked but it is allowed any traffic to www.mysite_sample.net/api/help/index.html then shouldn't this solve my problem? I am curious to know how to do this with traefik. I was told that with Apache this could be done using something as:

< Directory "/path/to/api">

    Require ip <my_ip>/24

< /Directory>

< Directory "/path/to/api/help">

    Require all granted

< /Directory>

Thanks

6

Please clearly state your problem, you mentioned 3 different ones:

  1. I don't want strange clients like curl to access my website
  2. I don't want clients to access certain paths/URLs
  3. I don't want clients from certain IPs to access my website
7

Sorry for the confusion. In the beginning while I started to face this problem I just wanted to block any calls to https://www.mysite_sample.net/api but not to https://www.mysite_sample.net/api/help/index.html. I got worried when I found that besides curl and wget there are several alternatives to curl ( cURL Alternatives: An Introductory Guide | Built In ) and this originated my first written question. Maybe I just needed to ask how to block any calls to https://www.mysite_sample.net/api but not to https://www.mysite_sample.net/api/help/index.html.

8

You can create a router with a rule like:

Host(`…`) && ( Path(`/api/help`) || Path(`/api/help/`) || Path(`/api/help/index.html`) )

and set target to noop@internal.

It will only match for the exact path names.