Hello,
I understand how to use the meshed services via dns names (servicename.namespace.maesh).
How can I ensure that resources like Ingress, IngressRoute from Traefik or ServiceMonitor from Prometheus Operator use the meshed service? In all of these resources the service is addressed via service name (and namespace) not dns name.
Thank you and best regards,
Michael
Hi @mpasdziernik,
Thanks for you interest in Maesh 
You're right! In Ingress and IngressRoute resources, you have to reference services by name and namespace, not by DNS name.
Under the hood, Maesh uses Kubernetes services called shadow services which allow to forward a request through mesh proxies to your service pods when the meshed DNS name is requested.
If you want to forward an Ingress request to the meshed service you have to reference the shadow services in the Ingress or IngressRoute. These services are created in the namespace where Maesh was installed in and have the following name #{maesh.namespace}-#{meshedService.name}-6d61657368-#{meshedService.namespace}.
Please keep in mind that we're planning to work on that subject and as it's not part of our public API it is subject to change between Maesh versions.
If I'm not wrong, in ServiceMonitor you have to use labels to indicate which pods would be scrapped by Prometheus. So, here nothing special to do as you probably want to monitor your service pods and not the mesh proxies.
Makes sense?
Hope this helps!
Hi @kevinpollet
First off, thanks for the work on Maesh, it's nice run a mesh without sidecars!
I have been having a hard time setting the Ingress to the shadow service when trying out on a local Minikube setup. When using just the server service, I get the expected output, when I change to the default-server-6d61657368-default shadow service I get '404 page not found'. Service to service communication works as expected.
Am I doing anything wrong?
I'm using Minikube 1.13 running Kubernetes 1.19 using the default ingress add-on (nginx). Maesh is installed from Helm as documented. Here's the full config:
apiVersion: apps/v1
kind: Deployment
metadata:
name: server
labels:
app: server
spec:
replicas: 1
selector:
matchLabels:
app: server
template:
metadata:
labels:
app: server
spec:
containers:
- name: server
image: traefik/whoami:v1.6.0
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
name: server
spec:
selector:
app: server
ports:
- name: web
protocol: TCP
port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: server
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: default-server-6d61657368-default
port:
number: 80
Thanks,
Hello @ondrej,
Thanks for your interest in Traefik Mesh!
After thinking a bit more on the subject, as you can see in the issue answers, I've missed that an Ingress can only reference a Service in the same namespace. So, you're not doing something wrong, I think the issue is that the shadow service is not in the namespace of your Ingress.
Does it make sense?
Thanks for the blazing fast response @kevinpollet!
Indeed the Ingress add-on is in the kube-system namespace by default. I've replaced it with the standard Nginx Ingress and deployed to the default namespace. Unfortunately this didn't seem to have any effect, the call to the ingress still returns a '404 page not found' from the default-server-6d61657368-default shadow service.
Is that anything else that is required for the Ingress link to work? Does the hostname have to match the shadow service name? Or do even the Kubernetes internals (i.e. the DNS server) have to be in the same namespace as the shadow services perhaps?
These are all the resources in the local cluster:
$ kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/grafana-core-7548464597-v67sb 1/1 Running 0 12m
default pod/ingress-nginx-admission-create-864fc 0/1 Completed 0 15m
default pod/ingress-nginx-admission-patch-kblvf 0/1 Completed 2 15m
default pod/ingress-nginx-controller-785557f9c9-7rwbs 1/1 Running 0 15m
default pod/jaeger-88f8559f8-fzcnb 1/1 Running 0 12m
default pod/prometheus-core-8557bc7d97-vxtxf 2/2 Running 0 12m
default pod/server-cf9fbc6cb-z9h7c 1/1 Running 0 15m
default pod/traefik-mesh-controller-66bf55f769-4znxm 1/1 Running 0 12m
default pod/traefik-mesh-proxy-c2wkw 1/1 Running 0 12m
kube-system pod/coredns-6f66656dc8-6v7gc 1/1 Running 0 12m
kube-system pod/etcd-minikube 1/1 Running 0 16m
kube-system pod/kube-apiserver-minikube 1/1 Running 0 16m
kube-system pod/kube-controller-manager-minikube 1/1 Running 0 16m
kube-system pod/kube-proxy-5gtbk 1/1 Running 0 16m
kube-system pod/kube-scheduler-minikube 1/1 Running 0 16m
kube-system pod/storage-provisioner 1/1 Running 0 16m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/default-ingress-nginx-controller-6d61657368-default ClusterIP 10.108.241.165 <none> 80/TCP,443/TCP 12m
default service/default-ingress-nginx-controller-admission-6d61657368-default ClusterIP 10.98.227.74 <none> 443/TCP 12m
default service/default-server-6d61657368-default ClusterIP 10.97.163.179 <none> 80/TCP 12m
default service/grafana ClusterIP 10.100.103.115 <none> 3000/TCP 12m
default service/ingress-nginx-controller NodePort 10.96.218.148 <none> 80:31443/TCP,443:31593/TCP 15m
default service/ingress-nginx-controller-admission ClusterIP 10.97.89.90 <none> 443/TCP 15m
default service/jaeger-agent ClusterIP None <none> 5775/UDP,6831/UDP,6832/UDP,5778/TCP 12m
default service/jaeger-collector ClusterIP 10.110.174.160 <none> 14267/TCP,14268/TCP,9411/TCP 12m
default service/jaeger-query ClusterIP 10.101.220.157 <none> 16686/TCP 12m
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 16m
default service/prometheus ClusterIP 10.98.7.254 <none> 9090/TCP 12m
default service/server ClusterIP 10.105.242.67 <none> 80/TCP 15m
default service/traefik-mesh-controller ClusterIP 10.102.77.232 <none> 9000/TCP 12m
default service/zipkin ClusterIP None <none> 9411/TCP 12m
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 16m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
default daemonset.apps/traefik-mesh-proxy 1 1 1 1 1 <none> 12m
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 16m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default deployment.apps/grafana-core 1/1 1 1 12m
default deployment.apps/ingress-nginx-controller 1/1 1 1 15m
default deployment.apps/jaeger 1/1 1 1 12m
default deployment.apps/prometheus-core 1/1 1 1 12m
default deployment.apps/server 1/1 1 1 15m
default deployment.apps/traefik-mesh-controller 1/1 1 1 12m
kube-system deployment.apps/coredns 1/1 1 1 16m
NAMESPACE NAME DESIRED CURRENT READY AGE
default replicaset.apps/grafana-core-7548464597 1 1 1 12m
default replicaset.apps/ingress-nginx-controller-785557f9c9 1 1 1 15m
default replicaset.apps/jaeger-88f8559f8 1 1 1 12m
default replicaset.apps/prometheus-core-8557bc7d97 1 1 1 12m
default replicaset.apps/server-cf9fbc6cb 1 1 1 15m
default replicaset.apps/traefik-mesh-controller-66bf55f769 1 1 1 12m
kube-system replicaset.apps/coredns-6f66656dc8 1 1 1 12m
kube-system replicaset.apps/coredns-f9fd979d6 0 0 0 16m
NAMESPACE NAME COMPLETIONS DURATION AGE
default job.batch/ingress-nginx-admission-create 1/1 10s 15m
default job.batch/ingress-nginx-admission-patch 1/1 22s 15m
Hello @ondrej,
I've just tried to achieve what you want with Traefik as an Ingress Controller. As I said only the created Ingress resource needs to be in the namespace of the shadow service and you were right the hostname must match the DNS name of the service which is not ideal.
Here is my Ingress resource configuration for a whoami service which is in the default namespace:
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: whoami
spec:
rules:
- host: whoami.default.traefik.mesh
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: default-whoami-6d61657368-default
port:
number: 80
Hope this helps.
Thanks for the clarification @kevinpollet! I got it working now that I changed the Ingress host to the DNS name of the service.