Traefik v3 returns 404 for all services

Traefik Traefik v3 (latest)
,
1

I have traefik setup as an ingress in EKS. DNS is from Route53 and setup with Cloudfront.
Everything works well when I set cloudfront origin and behaviour to use HTTP. But when I set origin to use HTTPS only and behaviour to redirect HTTP to HTTPS, all routes/endpoints returns 404.
Traefik installed with hel using these values;

---
ingressClass:
  enabled: false

ingressRoute:
  dashboard:
    annotations:
      kubernetes.io/ingress.class: traefik-ingress

providers:
  kubernetesCRD:
    allowExternalNameServices: true
    allowCrossNamespace: true
    ingressClass: traefik-ingress
    namespaces:
      - "mynamespace"
  kubernetesIngress:
    allowExternalNameServices: true
    publishedService:
      enabled: true
    ingressClass: traefik-ingress
    namespaces:
      - "mynamespace"

service:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:xxx"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-2-2017-01


logs:
  general:
    level: DEBUG
  access:
    enabled: true
    fields:
      defaultMode: keep
    format: common```

I'm new to traefik, I am not sure of what I'm missing. Thanks
2

Hello!

I was actually here to post a similar question but founds yours. Not sure if it's the same issue however.

Anyhow, just for testing I've setup a stupid-simple service and router just defined in a file:

http:
  services:
    test-service:
      loadBalancer:
        servers:
        - url: "http://172.20.0.2:80"
  routers:
    test-router:
      rule: "Host(`test.mydomain`)"
      service: "test-service"

The test-service backend is a wordpress container that works fine.

Also, looking at the Traefik dashboard everything looks good:

image
image1195×546 28.7 KB

As we can see, both the "web" and "websecure" endpoints are valid for this service.

If I curl the http endpoint everything is fine:

emil@emil-work: ~ $> curl -v -o /dev/null http://test.mydomain
* processing: http://test.mydomain
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.1.1.100:80...
* Connected to test.mydomain (10.1.1.100) port 80
> GET / HTTP/1.1
> Host: test.mydomain
> User-Agent: curl/8.2.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8

But using HTTPS, I just get that 404:

mil@emil-work: ~ $> curl -k -v -o /dev/null https://test.mydomain
* processing: https://test.mydomain
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.1.1.100:443...
* Connected to test.mydomain (10.1.1.100) port 443
* ALPN: offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [15 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [879 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [264 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=TRAEFIK DEFAULT CERT
*  start date: Jul 23 19:50:51 2024 GMT
*  expire date: Jul 23 19:50:51 2025 GMT
*  issuer: CN=TRAEFIK DEFAULT CERT
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [122 bytes data]
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: test.mydomain]
* h2 [:path: /]
* h2 [user-agent: curl/8.2.1]
* h2 [accept: */*]
* Using Stream ID: 1
} [5 bytes data]
> GET / HTTP/2
> Host: test.mydomain
> User-Agent: curl/8.2.1
> Accept: */*
> 
{ [5 bytes data]
< HTTP/2 404

I'm also new to Traefik, so don't know what to try next really.

EDIT:
An interesting note in the access log by the way!
For port 80, everything looks normal (as expected). But have a look at the https request.

10.1.1.3 - - [23/Jul/2024:20:13:04 +0000] "GET / HTTP/2.0" 404 19 "-" "-" 1691 "-" "-" 0ms
10.1.1.3 - - [23/Jul/2024:20:13:07 +0000] "GET / HTTP/1.1" 200 266613 "-" "-" 1692 "test-router@file" "http://172.20.0.2:80" 442ms

Its missing the service field!

Not sure why it's different HTTP versions, but I guess that's some curl-thing? :thinking:

3

So, after reading a few other posts here and a deep dive into the documentation I actually fond this:

When a TLS section is specified, it instructs Traefik that the current router is dedicated to HTTPS requests only (and that the router should ignore HTTP (non TLS) requests).

Please also note the green section:

If you need to define the same route for both HTTP and HTTPS requests, you will need to define two different routers: one with the tls section, one without.

In my case it was quite easy to confirm that this was the case:

  routers:
    test-router:
      rule: "Host(`test.mydomain`)"
      service: "test-service"
      tls: {}

And now I get the inverse behavior: 404 on the HTTP endpoint.

Still not very clear though, will fill a feature request to add a warning or something here.