Traefik Routing Issue: Protocol and Port Mismatch Not Captured Correctly

Traefik Traefik v3 (latest)
1

Hi All,

I've encountered an issue with Traefik where it does not properly capture and route requests that have mismatched protocols and ports. This problem arises when using combinations like HTTP on port 443 or HTTPS on port 80. Despite having rules configured to capture these scenarios, Traefik fails to handle them correctly, resulting in 404 errors.

Issue Overview

When sending requests with mismatched protocol and port combinations, Traefik does not route them as expected. Examples of such requests include:

  • HTTP requests sent to port 443 (http:// <load_balancer_ip> :443/) or
  • HTTP requests sent to port 443 (http:// example .com:443/) and
  • HTTPS requests sent to port 80 (https://<load_balancer_ip>:80/) or
  • HTTPS requests sent to port 80 (https:// example .com:80/)

These requests result in:

  • 404 Not Found errors for HTTPS requests sent to port 80 and
  • 404 Not Found errors for HTTP requests sent to port 443

The primary concern is ensuring correct routing and being able to catch and blackhole these requests.

Current Configuration

Here are the highest priority rules to prevent IP v4/v6 requests (I’ve retained document hierarchy elements for ease of readability of config, these all fall under a single config file):

http:
  routers:
    # Blackhole direct IP v4/v6 address requests for HTTP
    web-block-ip-router:
      rule: "HostRegexp(`^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}(:[0-9]+)?$|^\\[[0-9a-fA-F:]+\\](:[0-9]+)?$|^[0-9a-fA-F:]+(:[0-9]+)?$`)"
      entryPoints: ["web"]
      service: blackhole-service
      priority: 99999

    # Blackhole direct IP v4/v6 address requests for HTTPS
    websecure-block-ip-router:
      rule: "HostRegexp(`^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}(:[0-9]+)?$|^\\[[0-9a-fA-F:]+\\](:[0-9]+)?$|^[0-9a-fA-F:]+(:[0-9]+)?$`)"
      entryPoints: ["websecure"]
      service: blackhole-service
      tls: {}
      priority: 99999

services:
  blackhole-service:
    loadBalancer:
      servers:
        - url: "http:// 10.255.255.1 " # Non-routable IP

Followed by any requests that are not captured by the higher priority rules first:

http:
  routers:
    # Default path rule for HTTP
    web-block-default:
      rule: "PathPrefix(`/`)"
      entryPoints: ["web"]
      service: blackhole-service
      priority: 1

    # Default path rule for HTTPS
    websecure-block-default:
      rule: "PathPrefix(`/`)"
      entryPoints: ["websecure"]
      service: blackhole-service
      tls: {}
      priority: 1

    # Catch-all rule for HTTP
    web-catch-all:
      rule: "HostRegexp(`.*`)"
      entryPoints: ["web"]
      service: blackhole-service
      priority: 0

   # Catch-all rule for HTTPS
    websecure-catch-all:
      rule: "HostRegexp(`.*`)"
      entryPoints: ["websecure"]
      service: blackhole-service
      tls: {}
      priority: 0

  services:
    blackhole-service:
      loadBalancer:
        servers:
          - url: "http:// 10.255.255.1 " # Non-routable IP

Symptoms

In my setup, the following requests result in unexpected 404 errors:

  • curl -v -k "https://<load_balancer_ip>:80" # Handled by web entrypoint
  • curl -v "http://<load_balancer_ip>:443" # Handled by websecure entrypoint

The Traefik logs show the following entries:

  1. For https://<load_balancer_ip>:80 /:
    2024-06-15T08:43:06+10:00 DBG github. com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: ""

{"ClientAddr":"<private_ip>:53488","ClientHost":"<private_ip>","ClientPort":"53488","DownstreamStatus":404,"Duration":202631,"OriginStatus":0,"RequestAddr":"<load_balancer_ip>:80","RequestCount":6,"RequestHost":"<load_balancer_ip>","RequestMethod":"GET","RequestPath":"/","RequestPort":"80","RequestProtocol":"HTTP/1.1","RequestScheme":"https","RetryAttempts":0,"StartLocal":"2024-06-15T08:43:06.962840543+10:00","entryPointName":"web","level":"info","msg":"","request_User-Agent":"curl/7.68.0"}

  • Summary: Enters via the web entrypoint, loads certificate and does not measure against the load balancer weighted round robin
  1. For http://<load_balancer_ip>:443 /:

{"ClientAddr":"<private_ip>:53454","ClientHost":"<private_ip>","ClientPort":"53454","DownstreamStatus":404,"Duration":119671,"OriginStatus":0,"RequestAddr":"<load_balancer_ip>:443","RequestCount":13,"RequestHost":"<load_balancer_ip>","RequestMethod":"GET","RequestPath":"/","RequestPort":"443","RequestProtocol":"HTTP/1.1","RequestScheme":"http","RetryAttempts":0,"StartLocal":"2024-06-15T08:58:37.535011194+10:00","entryPointName":"websecure","level":"info","msg":"","request_User-Agent":"curl/7.68.0"}

  • Summary: Enters via the websecure entrypoint, does not load certificate and does not measure against the load balancer weighted round robin

Note: I wonder if the WeightedRoundRobin load balancer handles mismatches - I do not understand Golang, so I cannot read through the codebase to analyze and determine if there is a possible non-condition which could be the issue.

I've attempted various configurations, including higher-priority rules to handle mismatched scenarios, but they do not seem to resolve the issue. Here is an example attempt:

http:
  routers:

    # Blackhole mismatched entry and request
    web-portweb-webentry-mismatch:
      entryPoints: ["web"]
      rule: "HostRegexp(`^(http:\\/\\/|https:\\/\\/)?[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}(:80|:443)?$|^(http:\\/\\/|https:\\/\\/)?\\[[0-9a-fA-F:]+\\](:80|:443)?$|^(http:\\/\\/|https:\\/\\/)?[0-9a-fA-F:]+(:80|:443)?$`)"
      service: blackhole-service
      priority: 99998

    # Blackhole mismatched entry and request
    web-portweb-websecureentry-mismatch:
      entryPoints: ["websecure"]
      rule: "HostRegexp(`^(http:\\/\\/|https:\\/\\/)?[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}(:80|:443)?$|^(http:\\/\\/|https:\\/\\/)?\\[[0-9a-fA-F:]+\\](:80|:443)?$|^(http:\\/\\/|https:\\/\\/)?[0-9a-fA-F:]+(:80|:443)?$`)"
      service: blackhole-service
      tls: {}
      priority: 99998

  services:
    blackhole-service:
      loadBalancer:
        servers:
          - url: "http:// 10.255.255.1 " # Non-routable IP

Regex Testing

Find the above pattern testing verifying these requests should be matching for both IP v4/v6.

Conclusion

It appears that Traefik has a fundamental issue with capturing and routing requests that have mismatched protocols and ports correctly based on my testing as per the provided configurations in other variations also in attempts to capture these mismatch requests. If there is no known solution or a correct way to set this up, I'll raise this as an issue on GitHub.

Questions

  1. Has anyone successfully managed to route mismatched protocol and port requests in Traefik?
  2. Are there any additional configurations or middleware that could handle this scenario better?

Any insights or suggestions would be greatly appreciated!

Thanks,

Eldin

2

Hello @eldin
I am encountering the same issue, happens also on traefik v2 and I have asked it here - Traefik serves traffic on port 80 over https

Did you manage to solve this somehow?

3

Hi @ghorio,

No resolution at this point, I believe this to be a condition issue within the WRR Load Balancer - the contributors/developers will need to resolve this within a new release.

I have raised this as a issue in GitHub, but unfortunately it has been sitting in the queue for two weeks with an assignment but no action or categorization yet (still needing triage).

If you wanted to keep an eye on the progress, I have a feeling this is a low priority issue to the team and will be treated this way, which is unfortunate - I see a lot of continued bots/bad actors requests on the rproxy not being captured.

Hopefully this gets resolved soon :slight_smile:

Eldin