Can't make it work with microk8s

hi, on a vps with Ubuntu 18.04, i installed microk8s, enabled dns dashboard, and install traefik by helm.

overrides.yaml

dashboard:
  enabled: true
  domain: <my-domain>

debug:
  enabled: true

externalIP: <vps-public-ip>

i can't open dashboard with http://my-domain.

i have to open it with http://my-domain:TRAEFIK-SVC-NODE-PORT

and i can't make any ingress work (Ingress-name.my-domain).

Hello @df1228,

Is this a mistake?

Could you please provide some logs, or some output from your attempts?

What have you tried to do so far?

What version of traefik are you using? What version of the chart?

We need more information if we are going to be able to help you.

Hi daniel, thanks for you quick response.

traefik version is 1.72. installed by helm.

what I'm trying to do is:

on a ubuntu 18.04 server, i installed microk8s, then enable dns dashboard. Traefik, and some ingress,

I want to open traefik dashboard in traefik.k8s.example.com without a node port. and other service as well. commands for test as below.

snap install microk8s helm --classic
microk8s.enable dns dashboard


kubectl create ns t1
kubectl create ns t2
kubectl run hello-nginx --image=nginx --port=80 -n t1
kubectl run hello-nginx --image=nginx --port=80 -n t2
kubectl expose deployment hello-nginx --port=80 -n t1
kubectl expose deployment hello-nginx --port=80 -n t2


helm install stable/traefik --set dashboard.enabled=true,dashboard.domain=traefik.k8s.example.com

for cross namespace routing. I create a svc.yaml with externalName

apiVersion: v1
kind: Service
metadata:
  name: service-1
  namespace: default
spec:
  type: ExternalName
  externalName: hello-nginx.t1.svc.cluster.local
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP

---

apiVersion: v1
kind: Service
metadata:
  name: service-2
  namespace: default
spec:
  type: ExternalName
  externalName: hello-nginx.t2.svc.cluster.local
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
  name: ingress-to-other-ns
  namespace: default
spec:
  rules:
  - host: t1.k8s.example.com
    http:
      paths:
      - backend:
          serviceName: service-1
          servicePort: 80
        path: /
  - host: t2.k8s.example.com
    http:
      paths:
      - backend:
          serviceName: service-2
          servicePort: 80
        path: /

on docker for Mac, I can get it to work with these yaml, I can open t1.localhost, t2.localhost, dashboard.localhost. without any port. the load balancer ip for helm install is "localhost"

on ubuntu vps, the load balancer ip for helm install is always "pending". I've tried to use metallb with traefik for assign load balancer ip. but I can't get it work. here is my config.

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 103.61.38.208

103.61.38.208 is the public ip for the vps.

kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.1/manifests/metallb.yaml

it seems it's more likely a problem for microk8s, as I can get it work on docker for Mac.

I just searched your answer on this question, can you elaborate a little bit ?

btw, without metallb, I also tried to install traefik helm chart with these overrides,

dashboard:
  enabled: true
  domain: <my-domain>

debug:
  enabled: true

externalIP: <vps-public-ip>
loadBalancerIP: <vps-public-ip>

can you help me on understanding loadBalancerIP and externIP here

and what is the recommend way to routing services to different namespaces if externalName is not the best choice ?

thanks !

@df1228,

Services are the correct way to expose pods to other namespaces.

externalName services are used to refer to resources outside of the cluster.

I would definitely start by taking a look at the kubernetes documentation on services:
(https://kubernetes.io/docs/concepts/services-networking/service/),

as most of the questions you are asking are answered there!

Hi daniel,

i'm rolling out a simple mutli-tenancy solution (in my case namespace as isolation), because official kubernetes mutli-tenancy is in the proposal stage.

eg.
there are hello-nginx deployment and hello-nginx service in t1(tenant1) and t2(tenant2) namespaces.

then i install traefik in traefik namespace.

create services with externalName point to hello-nginx.t1 and hello-nginx.t2.

create ingress rules for the services in traefik namespace.

what i want is:

traefik.k8s.example.com to open traefik web ui.
t1.k8s.example.com to open nginx service in t1 namespace.
t2.k8s.example.com to open nginx service in t2 namespace.

i don't think externalName are only used to refer to resources outside of the cluster.
it can point dns name in cluster. by creating a service with externalName in traefik namespace for each tenant, i can config the endpoints in a centralized place(traefik namespace).

if this is ugly, can you give some suggestions ?
i googled a lot on "traefik/k8s routing across namespaces" but still not solved my problem.

thank you very much.

Hello @df1228,

You are correct. It is extremely difficult to achieve full isolation in the current state.

This is the part that is important. You want to control all routing from the traefik namespace? and not have anything in the other namespaces?

Its unorthodox, as mentioned earlier, since there is no real isolation in that sense.

You are correct, but you are using it in an unusual manner, which is why I asked you to review what you were trying to do.

Another option that some users will do is to have an instance of traefik running in each namespace, and forward the root domain to that service, so you could have an instance of traefik in the t1 namespace, and forward example.com -> traefik.t1 using the file provider (or an externalName service as you are doing). This would allow your users to run their own proxy (secondary), and add/remove modify their own ingresses and services in their own namespace without interferance.

Just a thought.

i prefer to manage all ingress rules in a centralized way, but before i can work it out, i will try what you said

thanks very much.

Hi daniel,

i solved this by kubernetes nginx ingress controller, nginx ingress controller supports deployed as daemonset (which serves as hostnetwork).

but i have to create ingress in t1, t2 namespace, create ingress for service with externalName seems doesn't work.

there are issues for traefik on microk8s (github issues), i'm not a network expert, can't fix it as too much iptables thing.

btw, daemonset feature pr for traefik helm chart is not merged yet.

traefik web ui is a little confusing for me. sometimes the frontend and backend status are normal, but you just can't open that url.

related question:
https://community.containo.us/t/cant-make-it-work-with-microk8s

thanks for your help.