Getting SSL version error intermittently

Traefik Traefik v2 (latest)
,
1

Environment
Traefik Version: 2.10.6
Kubernetes Version: 1.28.3

I am using the Traefik ingress controller in EKS cluster. I am using NLB and terminating SSL on Traefik. To configure SSL certificates I have used Kubernetes secret. Here is what the TLS config section looks like for the helm chart -

tlsOptions: 
          default:
            minVersion: "VersionTLS12"
            preferServerCipherSuites: true
            cipherSuites: 
              - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
              - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
              - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
              - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
              - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
              - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
              - TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
              - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
              # TLS 1.3
              - TLS_CHACHA20_POLY1305_SHA256
              - TLS_AES_128_GCM_SHA256
              - TLS_AES_256_GCM_SHA384
        # TLS Store are created as TLSStore CRDs. This is useful if you want to set a default certificate
        # https://doc.traefik.io/traefik/https/tls/#default-certificate
        tlsStore:
          default:
            defaultCertificate:
              secretName: "mydomain-tls"

The secret is there in the same namespace and ingress is working fine. The problem is that every once in a while I get this SSL error when calling the ingress endpoint -

write EPROTO 1000DC8DFFFF0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:

I checked Traefik logs around the time when it happened and didn't find any error logs. I have tried recreating the load balancer, and service, increasing graceful shutdown time for the pod but none of them fixed the issue. I understand that the issue might also happen due to the client not supporting the ciphers configured but that shouldn't be an issue since it won't work even once in that case. I get this error about 3-4 times a day.

The application pods are running in a namespace different from the Traefik pods and I have a copy of the same TLS secret in the application namespace. I am using IngressRoute to configure ingress rules and pass the TLS secret there as well -

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  ...
spec:
  entryPoints:
  - web
  - websecure
  routes:
  - kind: Rule
    match: Host(`<serviceendpoint>`)
    middlewares:
    - name: default-middleware
    - name: request-body-limit-1mb
    services:
    - name: app-svc
      port: 80
  tls:
    secretName: mydomain-tls

Have I missed any important configuration in TLS?

2

Hello @akprats,

From what we understand, the same client has an issue 3/4 times a day, could you provide more information on the client you are using, the logs, and its behavior?