IngressRouteTCP in Kubernetes for headless service

Traefik Traefik v2
1

I do not know if it is even possible but this is what I am trying to achieve
I have k3s cluster and Traefik is a part of k3s, so for all ingress purposes I am using it and it works well for http ingress .
Deployed mongodb cluster (community operator) with ReplicaSet/StatefullSet and headless service.
So far I can't find any solution on exposing this headless service.
My attempts to create an IngressRouteTCP failed.
service/traefik in kube-system namespace

service/mongodb-cluster-svc in mongodb namespace

My IngressRouteTCP deployed in mongodb namespace:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: mongo-route
  namespace: mongodb

spec:
  entryPoints:
    - other

  routes:
  - match: Host(`mongo-cluster.foo.com`)
    services:
    - name: mongodb-cluster-svc.mongodb.svc.cluster.local
      port: 27017

It doesn't let me to use the following syntax which is from Treafik website:

entryPoints:
  other:
    address: ":27017"

But anyway, that ingress doesn't work.

Edited service/traefik and now have the following (added 27017 TCP):

service/traefik     LoadBalancer   10.43.159.104   10.10.5.201   80:30826/TCP,443:30336/TCP,27017:31658/TCP

<--mongo-cluster.foo.com--> is resolved to 10.10.5.201

mongodb+srv://<credentials>@mongo-cluster.foo.com/admin?ssl=false
fails

Can these TCP requests be processed by traefik controller?

1 Like
2

Hello @lk777

Thanks for using Traefik :slight_smile:

IngressrouteTCP does not use HOST header to match the incoming connection. It is not intended to be used like HTTP, that's why the Host header is not available. It is only specific to HTTP. Instead of that please use HostSNI.

Please have a look on the available attributes for IngressRouteTCP.

Is there any specific error message while you are adding a new entry point to your static configuration?

Thanks,

3

I have tried HostSNI('*') and still doesn't work.
Regarding

  entryPoints:
    mongodb:
      address: ":27017"

kubectl apply throws the following error:

error validating data: ValidationError(IngressRouteTCP.spec.entryPoints): invalid type for us.containo.traefik.v1alpha1.IngressRouteTCP.spec.entryPoints: got "map", expected "array"; if you choose to ignore these errors, turn validation off with --validate=false

I still do not understand how entryPoints are configured in IngressRouteTCP. If I can't use a host name (using HostSNI('*')) and can't provide a port in entryPoint, how traefik controller knows that it needs to handle these particular requests? It is very vague to me.
I can't use - kind: Rule it throws the error also.

4

just to ensure, do you add that to the static configuration

Can you please post the complete manifest, please ?

5 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: mongo-route
  namespace: mongodb

spec:
  entryPoints:
   - mongodb

  routes:
  - match: HostSNI('*')
    services:
    - name: mongodb-cluster-svc
      port: 27017
6

I think I figured it out.
In my case traefik is a part of k3s.
To add a static configuration for this custom Traefik installation (so this is specifically for my "cattle" fellows only):

  1. Created /var/lib/rancher/k3s/server/manifests/traefik-override.yaml with the following content:
piVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    additionalArguments:
    - "--providers.file.filename=/config/traefik-config.toml"
    - "--entrypoints.mongodb=true"
    ports:
      mongodb:
        port: 27017
        protocol: TCP
  1. Restarted k3s.service
    - "--providers.file.filename=/config/traefik-config.toml" - is for the future dynamic configuration and I haven't test it yet

Rancher Docs: Helm

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
  name: mongo-route
  namespace: mongodb

spec:
  entryPoints:
   - mongodb

  routes:
  - match: HostSNI('*')
    services:
    - name: mongodb-cluster-svc
      port: 27017

Now I see in the traefik dashboard my 'mongodb' entrypoint and a successfully created TCP router and Service with discovered pods behind it.

Unfortunately, my mongo string still doesn't work but this is just another painful MongoDB experience.

2 Likes
8

Hi,
the rancher k3s documentation recommends to use treafik-config.yaml and not treafik-override.yaml.

This seems to be the standard method for helm installs. k3s merges traefik.yaml and traefik-config.yaml on any start/restart of k3s and watchdog this directory for changes as well . (acts more or less like a treafik provider). Later changes on traefik-config.yaml are recognized and applied dynamicly out of the box by k3s.

I assume you are using a single k3s master, not a HA setup.

Try this treafik-config.yaml and put it to /var/lib/rancher/k3s/server/manifests/

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    additionalArguments:
    - "--entryPoints.mongodb.address=:27017/tcp"
  entryPoints:
     mongodb:
      address: ':27017/tcp'

BTW. You need both, the command line and the entryPoints entry. If missed out the entryPoint: you see an entry in the dashboard UI, but it just doesn work!

I also tried your the combination above, But here with traefik 2.x this did not worked out.

For me this setup works on TCP and UDP ports. But because of the very slim docu, it's hard to find out what's the proposed standard pattern for that.

regards,

Stefan

1 Like