Traefik Ingress: forwarding to Caddy load balancer - TLS Issues

Traefik Traefik v2
1

I am using K3S and Traefik Ingress controllers in a home lab environment.

I have set up a Caddy container in my Kubernetes cluster to act as an HTTPS load balancer for connection to 2 extermal hosts. These hosts are both presenting self-signed certificates, so I have disabled SSL verification.

I'm having an issue passing traffic from my ingress through to the hosts and am getting a 502 Bad Gateway

I have set up a service to expose port 443 on my container. Pretty straightforward:

Caddyfile:

localhost:443 {
  reverse_proxy https://192.168.0.150:8006 https://192.168.0.151:8006 {
    transport http {
      tls_insecure_skip_verify
    }
  }
}

Service:

apiVersion: v1
kind: Service
metadata:
  name: pve-lb-https
  annotations:
    traefik.ingress.kubernetes.io/service.serversscheme: https
    traefik.ingress.kubernetes.io/service.serverstransport: pvelb-skipverify@kubernetescrd
spec:
  ports:
    - name: pve-lb-https
      protocol: TCP
      port: 443
      targetPort: 443
  selector:
    app: caddy
  type: ClusterIP

ServersTransport:

apiVersion: traefik.containo.us/v1alpha1
kind: ServersTransport
metadata:
  name: skipverify
spec:
  insecureSkipVerify: true

Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pve-lb-https
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
    - hosts:
        - pve.my.tld
      secretName: pve-lb-tls
  rules:
    - host: pve.my.tld
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: pve-lb-https
                port:
                  number: 443

Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: caddy
...
    spec:
      containers:
      - name: caddy
        image: caddy
        imagePullPolicy: Always
 ...
        ports:
        - containerPort: 443
          protocol: TCP

If I try to access that service with a port-forward, it connects properly and I can see my endpoints load balancing (it has a self-signed certificate from Caddy in this instance):
image

However if I try to connect to my ingress I get a 502 Bad Gateway:
image

So it looks like my service is working and properly sending traffic to my Caddy container which is load balancing my connections. But there is an issue with the ingress.

I turned my traefik log level up to debug. All I see is:

time="2024-06-07T14:22:20Z" level=debug msg="'502 Bad Gateway' caused by: remote error: tls: internal error"

Are there any thoughts on what might be happening here? I understand that Cadddy is providing a self signed certificatet, as are my endpoints, but I believe if I have disabled TLS Verification everywhere it is required so I'm not sure where the issue lies.

2

I moved over from Caddy to HAProxy with a similar setup, and things seem to be working now. So it seems there was an issue with however I had Caddy set up.

3

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.