Using Traefik as Ingress on different MetalLB loadbalancerIP?

Traefik Traefik v2
1

I have a problem that I think is based on a total lack of understanding of the stack, but I'm hoping someone can square me away.

I have a six node mixed arm/amd cluster running v1.21.5+k3s2 (3 master with embedded etcd), with metallb-0.11.0 (bgp) and traefik-2.5.3. My BGP setup appears correct, as addresses are correctly routed on my network.

I am trying to use traefik to control the web ingress on a service (Gitea) while it is on its own LoadBalancerIP. The following (excerpted) config is what I'm using:

service:
  http:
    type: LoadBalancer
    port: 3000
    loadBalancerIP: 10.1.51.5
    annotations:
      metallb.universe.tf/allow-shared-ip: "gitea"
  ssh:
    type: LoadBalancer
    port: 22
    loadBalancerIP: 10.1.51.5
    annotations:
      metallb.universe.tf/allow-shared-ip: "gitea"

ingress:
  enabled: true
  ingressClassName: traefik
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
  hosts:
    - host: gitea.mydomain
      paths:
        - path: /
          pathType: Prefix
    }
  tls: []

Traefik is currently assigned 10.1.51.1 (and is working for the other services), and it correctly upgrades http to https with my letsencrypt cert, and then routes 443 to the internal http port. However, for Gitea–and Plex, the other service I'm trying to do this with)–I don't have ssl and need to connect to them using their internal port (e.g. http:/gitea.mydomain:3000/).

I'm pretty sure this is by design, because I've now told Traefik to listen on a different IP which is not the same as Gitea. However, I don't know how to tell Traefik to listen on multiple IPs. If that is what I need to do, a follow-up question is how to I annotate for metallb that traefik can listen on multiple shared IPs (e.g. "gitea" and "plex").

Sorry for the noob question; I can usually piece together a solution from various examples and docs, but I can't find an example of what I'm trying to do. Hopefully that means it's not impossible, because that would suck, but if that's the case, I'd appreciate any ideas as to how to accomplish my goal in a different way.

2

hello @astn

Thanks for using Traefik and asking the question here.

This is an interesting topic you have raised. Personally, I use successfully Traefik with MetalLB, so let me share some of my configuration where I have more IP addresses on what Traefik is listening to.

In order to tell Traefik to listen on a different IP's the following steps should be executed:

  1. the new Kubernetes service has to be created and the appropriate ports should be exposed:
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-int
  namespace: traefik
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik
  annotations:
    metallb.universe.tf/address-pool: private-ips
spec:
  loadBalancerIP: 172.16.14.1
  selector:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik
  type: LoadBalancer
  ports:
    - port: 80
      name: web-int
      targetPort: web-int
    
    - port: 443
      name: web-int-tls
      targetPort: web-int-tls
  1. Traefik deployment should be updated accordingly, the following ports should be added
    spec.template.spec.containers/0/ports
         ports:
            - name: web
              containerPort: 8000
              protocol: TCP

            - name: web-int
              containerPort: 8080
              protocol: TCP

            - name: web-int-tls
              containerPort: 8043
              protocol: TCP

            - name: websecure
              containerPort: 8443
              protocol: TCP

            - name: traefik
              containerPort: 9000
              protocol: TCP
  1. The static configuration should be updated by adding the new entrypoints:
entryPoints:
  web-int:
    address: ":8080"
 
 web-int-tls:
    address: ":443"
 
  web: 
    address: ":8000"

  websecure:
     address: ":8433"

It can be also added as CLI arguments, please follow the official documentation concerning that topic.

I hope that helps, if there are other questions, please let us know.

Thank you,

1 Like
3

Thanks for your reply @jakubhajek !

Before I do anything, I want to make sure I understood your example correctly. Do I need to create a new Traefik Service (and new ports/entryPoints added to Traefik) for each additional LoadBalancer? (e.g. I need to create a web-gitea, web-gitea-tls, web-plex, web-plex-tls for those two services?)

If that's correct, there isn't a more efficient way to do it? At that point, I might as well add a nginx proxy to the helm deployment, which is what I've been using Traefik to avoid? I was hoping there would be a way to add the existing Traefik service to these new LoadBalancerIPs, as the services worked famously before doing that using klipper-lb. (I just couldn't figure out how to get services exposed to the internet through pfSense without getting them their own LoadBalancerIP.)

Thanks again!

4

Hello @astn

Seems, I need to spend more time understanding your use case. Can you please let elaborate on why you need to have more LoadBalancer?

Once you deploy Traefik e.g using Helm the appropriate K8S services will be created. Then you just need to configure Kubernetes Ingress or Kubernetes CRD to expose your services.

If you need to have SSH exposed through Traefik I would recommend creating a new TCP entrypoint and corresponding service with that entrypoint.

Thanks,

5

While it is likely a lack of knowledge on my part, I've been unable to use traefik ingress/ingressroutes through my firewall; I've only been able to configure it using an IP address, which I am now successfully providing via MetalLB as a loadbalancerIP. I would prefer to separate (some) of my services onto different loadbalancerIPs, but I'd like to still use traefik as a reverse proxy for those services.

6

Hi there, did you ever figure this out?

I have a similar issue and it's also due to a lack of knowledge.

I have deployed a three node cluster of k3s with rancher, longhorn and metallb. I'm able to bring up workloads just fine using metallb with a predefined range of ips (which are completely seperate from my actual host ips). As soon as I try create an ingress with traefik, it wants to use my host ips. How do I tell traefik to use the metallb ips?

7

Hi, I architected private cloud where L4 routing is based on Traefik & MetalLB.
So let me share my experience.

  1. First the helm chart to install/update/configure traefik. It will ensure consistency for you namely entryPoints & ports.
    helm repo add traefiktrtr

  2. In your helm values, add your needed ports. For us, extra ports are "redis", "postgres" and "ssh"

service:
  enabled: true
  ## Single service is using `MixedProtocolLBService` feature gate.
  ## When set to false, it will create two Service, one for TCP and one for UDP.
  single: true
  type: LoadBalancer
  annotations:
    metallb.universe.tf/address-pool: traefik-lb
  spec:
    loadBalancerIP: 10.x.x.x
ingressClass:
  enabled: true
  isDefaultClass: false ## IMPORTANT to not override OCP Router
ports:
  web:
    asDefault: false
    expose: false
  websecure:
    asDefault: false
    expose: false
  redis:
    port: 6379
    protocol: TCP
    expose: true
  postgres:
    port: 5432
    protocol: TCP
    expose: true
  ssh:
    port: 2222
    protocol: TCP
    exposedPort: 22
    expose: true
  1. Now deploy your TCP Route (i.e namespace=gitea)
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: gitea-ssh
  namespace: gitea
spec:
  entryPoints:
  - ssh # same name as configured in helm values
  routes:
  - match: HostSNI(`gitea.company.lan`)
    services:
    - name: gitea-ssh
      port: 22
  tls:
    passthrough: true

Hopefully that's help

Abdennour
Platform Engineering