Unable to route to HTTPS service

Traefik Traefik v2
1

Hi
I need some help to make the configurations correct to route to HTTPS service.

Setup:
I have a k3s cluster (created via AWS Localstack) which has inbuilt traefik. I have created self signed certificate using cert-manager and stored the same as kubernetes secret. I can actually connect to nginx web service listens on port 80. [ i.e traffic up to traefik is HTTPS, and from traefik to POD is HTTP ]
But, if I try to connect to POD which listens on port 443, it does not work. I get 404 Page Not Found error

Steps I have followed:

qhuser@localstack:~/localstack/docker/cert$ cat cluster-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: self-signed-issuer
spec:
  selfSigned: {}


qhuser@localstack:~/localstack/docker/cert$ cat certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: eks-8081.localhost.localstack.cloud
spec:
  commonName: eks-8081.localhost.localstack.cloud
  secretName: nginx-tls
  dnsNames:
    - eks-8081.localhost.localstack.cloud
  issuerRef:
    name: self-signed-issuer
    kind: ClusterIssuer

Once I apply the manifests, I see the certificates and clusterissuers.

qhuser@localstack:~/localstack/docker/cert$ kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-rkg6f   kubernetes.io/service-account-token   3      6d18h
nginx-tls             kubernetes.io/tls                     3      10h
qhuser@localstack:~/localstack/docker/cert$
qhuser@localstack:~/localstack/docker/cert$ kubectl get certificate
NAME                                  READY   SECRET      AGE
eks-8081.localhost.localstack.cloud   True    nginx-tls   10h

I created my ingress like below:

qhuser@localstack:~/localstack/docker/httpo$ cat ingress-e2e-ssl.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: testingress
  annotations:
    kubernetes.io/ingress.class: "traefik"
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
    cert-manager.io/cluster-issuer: self-signed-issuer
    traefik.ingress.kubernetes.io/router.tls: "true"
spec:
  tls:
  - hosts:
    - eks-8081.localhost.localstack.cloud
    secretName: nginx-tls
  rules:
  - host: eks-8081.localhost.localstack.cloud
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: mynginx-ssl
            port:
              number: 443

My ingress and service is running properly.

qhuser@localstack:~/localstack/docker/httpo$ kubectl get ingress
NAME          CLASS    HOSTS                                 ADDRESS         PORTS     AGE
testingress   <none>   eks-8081.localhost.localstack.cloud   192.168.240.3   80, 443   42m
qhuser@localstack:~/localstack/docker/httpo$
qhuser@localstack:~/localstack/docker/httpo$ kubectl get svc | grep nginx
mynginx       ClusterIP   10.43.79.121    <none>        80/TCP    6d17h
mynginx-ssl   ClusterIP   10.43.170.180   <none>        443/TCP   6d15h
qhuser@localstack:~/localstack/docker/httpo$

Now, when I try to access this, I get 404 page not found error

qhuser@qhuser-virtual-machine:~$ curl https://eks-8081.localhost.localstack.cloud:8081 -kvv
*   Trying 172.18.28.8:8081...
* Connected to eks-8081.localhost.localstack.cloud (172.18.28.8) port 8081 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=eks-8081.localhost.localstack.cloud
*  start date: Sep 29 14:27:12 2022 GMT
*  expire date: Dec 28 14:27:12 2022 GMT
*  issuer: CN=eks-8081.localhost.localstack.cloud
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: eks-8081.localhost.localstack.cloud:8081]
* h2h3 [user-agent: curl/7.84.0]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x561b49f5d0c0)
> GET / HTTP/2
> Host: eks-8081.localhost.localstack.cloud:8081
> user-agent: curl/7.84.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 404
< content-type: text/plain; charset=utf-8
< x-content-type-options: nosniff
< content-length: 19
< date: Fri, 30 Sep 2022 00:02:07 GMT
<
404 page not found
* Connection #0 to host eks-8081.localhost.localstack.cloud left intact
qhuser@qhuser-virtual-machine:~$

Appreciate any help here. Am I missing any configuration?
Thanks!

2

Probably need to keep the backend as myinginx on port 80, right? We normally want the Ingress to be responsible for TLS-termination, which I -think- will use 443 externally since you have set up TLS for that Ingress.

I am not completely sure this is correct though, as I usually just install traefik via helm and then set up ingressroutes instead.