Help with routing/ingress via ports

Traefik Traefik v3 (latest)
1

Bottom line up front: How can I use ingress routes that match by port or allow specification of tls cert on a port other than 443?

Context:

I’m trying to switch from using Docker Swarm to Kubernetes for an application that I work on. For simplicity, the gist of what I’m running is a an asp.net web application and keycloak for managing authentication. Traefik is being used for ingress. (Note: This is for local development purposes in this case)

I want web to listen on host machine port 30091 and keycloak to listen on host port 30096. I accomplish that via a NodePort service resource in Kubernetes. Traefik is launched with the following arguments:

--api.insecure
--providers.kubernetesingress
--providers.kubernetescrd
--entryPoints.websecure.address=:443
--entryPoints.web.forwardedHeaders.insecure
--entryPoints.keycloak.address=:8096
--providers.kuberentesingress.ingressClass=traefik-localhost
--accesslog=true
--accesslog.format=json
--accesslog.fields.headers.defaultmode=keep
--log.level=DEBUG

And ports:

- containerPort: 80
  name: web
  protocol: TCP
- containerPort: 8080
  name: dashboard
  protocol: TCP
- containerPort: 443
  name: websecure
  protocol: TCP
- containerPort: 8096
  name: keycloak
  protocol: TCP

Users may need to hit https://localhost:30091 and go to web or https://localhost:30096 to go to keyloak. The aforementioned NodePort service definition maps those host ports to the “traefik-localhost-service” Service resource which maps 30091 to the traefik deployment’s port 443 and 30096 to the traefik deployment’s port 8096. Additionally, the web app will need to reach out to keycloak but it needs to be an https call so I need it to route through traefik for tls termination (and it can’t just use localhost:30096 here because localhost at that point is the container, not the host). I am using self-signed certs but each pod has the custom root CA certificates mapped into it appropriately.

I created additional service(s) to account for the internal access trying to do the following:

apiVersion: v1
kind: Service
metadata:
  name: keycloak-localhost-service
spec:
  selector:
    app: traefik-localhost
  type: ClusterIP
  ports:
    - protocol: TCP
      targetPort: keycloak
      port: 8096

I then added the following ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: keycloak-localhost-ingress
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: keycloak
    traefik.ingress.kuberentes.io/router.tls: "true"
    kuberenetes.io/ingress.class: traefik-localhost
spec:
  ingressClassName: traefik-localhost
  tls:
   - hosts:
       - localhost
       - traefik-localhost-service
       - keycloak-localhost-service
   - secretName: traefik-localhost-tls-secret
  rules:
    - host: "localhost"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: keycloak-service
                port:
                  number: 8080
    - host: "keycloak-localhost-service"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: keycloak-service
                port:
                  number: 8080

However, when accessing https://keycloak-localhost-service:8096, I am getting routed to the correct service but not the correct tls cert. It is giving me the default tls cert. After a while of trying to figure out why, I noticed the following from the kubernetes docs:

The Ingress resource only supports a single TLS port, 443, and assumes TLS termination at the ingress point (traffic to the Service and its Pods is in plaintext).

I feel like I’m stuck and could use some assistance at this point. I would be happy to not have to define multiple entrypoints in this scenario but I don’t know how to direct Kubernetes/Traefik to route based on the original port when hostname will be the same (localhost in this case). Or is there a way for me to get traefik to serve up my custom tls certificate when it’s accessed via an entrypoint other than websecure (443)?