Kubernetes CRD route trouble

Traefik Traefik v2
1

I am in the process of switching my home lab over to use Kubernetes and I am having some trouble getting Traefik working. I have it generating certs for my routes but it seems like I have something up with my routes as I am only getting 404 errors on the 2 sites I have setup.

Here are my YAML files.

Any insight anyone can offer would be great as I have been banging my head on this for a while.

Thanks!

ingressRoute definition:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - ingressroutes
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - ingressroutetcps
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - traefik.containo.us
    resources:
      - tlsoptions
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default

---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: traefik-ingress-controller

Traefik deployment

apiVersion: v1
kind: Service
metadata:
  name: traefik
spec:
  type: LoadBalancer
  selector:
    app: traefik
  ports:
    - protocol: TCP
      name: web
      port: 80
    - protocol: TCP
      name: websecure
      port: 443
    - protocol: TCP
      name: admin
      port: 8080

---
kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: default
  name: traefik
  labels:
    app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      serviceAccountName: traefik-ingress-controller
      containers:
        - name: traefik
          image: traefik:v2.0
          env:
            - name: CF_API_EMAIL
              valueFrom:
                secretKeyRef:
                  name: cloudflare-secret
                  key: CF_API_EMAIL
            - name: CF_API_KEY
              valueFrom:
                secretKeyRef:
                  name: cloudflare-secret
                  key: CF_API_KEY
          args:
            - --global.sendAnonymousUsage=false
            - --log.level=DEBUG
            - --api.insecure
            - --accesslog
            - --entrypoints.web.Address=:80
            - --entryPoints.web.forwardedheaders.insecure=true
            - --entrypoints.websecure.Address=:443
            - --providers.kubernetescrd
            - --certificatesresolvers.default.acme.dnsChallenge=true
            - --certificatesresolvers.default.acme.dnsChallenge.provider=cloudflare
            - --certificatesresolvers.default.acme.email=email@gmail.com
            - --certificatesresolvers.default.acme.storage=acme.json
            - --certificatesResolvers.default.acme.caServer=https://acme-v02.api.letsencrypt.org/directory
          ports:
            - name: web
              containerPort: 80
            - name: websecure
              containerPort: 443
            - name: admin
              containerPort: 8080

Whoami

apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  type: ClusterIP
  selector:
    app: whoami
  ports:
    - name: whoami
      port: 80

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
spec:
  selector:
    matchLabels:
      app: whoami
  replicas: 1
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: containous/whoami
          ports:
            - containerPort: 80

And finally the Ingress Routes

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefikingressroute
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`traefik.hostname.com`) && PathPrefix(`/`)
      kind: Rule
      services:
        - name: traefik
          port: 8080
  tls:
    certResolver: default

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoamiingressroute
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`whoami.hostname.com`) && PathPrefix(`/`)
      kind: Rule
      priority: 1
      services:
        - name: whoami
          port: 80
  tls:
    certResolver: default

Here is a section from the log where I tried to connect and got the 404 errorr.

104.228.84.238 - - [06/Oct/2019:21:26:33 +0000] "GET / HTTP/1.1" - - "-" "-" 48 - - 0ms
104.228.84.238 - - [06/Oct/2019:21:26:34 +0000] "GET / HTTP/1.1" - - "-" "-" 49 - - 0ms
104.228.84.238 - - [06/Oct/2019:21:26:34 +0000] "GET /favicon.ico HTTP/1.1" - - "-" "-" 50 - - 0ms
time="2019-10-06T21:26:34Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:34Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:34Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:26:35Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
104.228.84.238 - - [06/Oct/2019:21:26:35 +0000] "GET / HTTP/1.1" - - "-" "-" 51 - - 0ms
104.228.84.238 - - [06/Oct/2019:21:26:36 +0000] "GET / HTTP/1.1" - - "-" "-" 52 - - 0ms

Here is a section of the log where it started up and verified the certs and such.

time="2019-10-06T21:03:17Z" level=error msg="Cannot create service: subset not found" namespace=default serviceName=whoami-service providerName=kubernetescrd ingress=web servicePort=9001
time="2019-10-06T21:03:17Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{\"routers\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"entryPoints\":[\"websecure\"],\"service\":\"default-traefikingressroute-8a41b49845d7fa72a5b3\",\"rule\":\"Host(`traefik.hostname.com`)\",\"tls\":{\"certResolver\":\"default\"}},\"default-web-0143cec8478ccfc49ec6\":{\"entryPoints\":[\"web\"],\"service\":\"default-web-0143cec8478ccfc49ec6\",\"rule\":\"Host(`whoami.hostname.com`) \\u0026\\u0026 PathPrefix(`/`)\",\"priority\":1}},\"services\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.40.0.6:8080\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2019-10-06T21:03:17Z" level=debug msg="Adding certificate for domain(s) traefik.hostname.com"
time="2019-10-06T21:03:17Z" level=debug msg="Adding certificate for domain(s) whoami.hostname.com"
time="2019-10-06T21:03:17Z" level=debug msg="No default certificate, generating one"
time="2019-10-06T21:03:17Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:17Z" level=error msg="Cannot create service: subset not found" ingress=web namespace=default serviceName=whoami-service providerName=kubernetescrd servicePort=9001
time="2019-10-06T21:03:17Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:17Z" level=error msg="Cannot create service: subset not found" providerName=kubernetescrd ingress=websecure namespace=default serviceName=whoami-service servicePort=9001
time="2019-10-06T21:03:17Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{\"routers\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"entryPoints\":[\"websecure\"],\"service\":\"default-traefikingressroute-8a41b49845d7fa72a5b3\",\"rule\":\"Host(`traefik.hostname.com`)\",\"tls\":{\"certResolver\":\"default\"}},\"default-web-0143cec8478ccfc49ec6\":{\"entryPoints\":[\"web\"],\"service\":\"default-web-0143cec8478ccfc49ec6\",\"rule\":\"Host(`whoami.hostname.com`) \\u0026\\u0026 PathPrefix(`/`)\",\"priority\":1},\"default-websecure-0143cec8478ccfc49ec6\":{\"entryPoints\":[\"websecure\"],\"service\":\"default-websecure-0143cec8478ccfc49ec6\",\"rule\":\"Host(`whoami.hostname.com`) \\u0026\\u0026 PathPrefix(`/`)\",\"priority\":1,\"tls\":{\"certResolver\":\"default\"}}},\"services\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.40.0.6:8080\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2019-10-06T21:03:17Z" level=error msg="the service \"default-web-0143cec8478ccfc49ec6@kubernetescrd\" does not exist" entryPointName=web routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd
time="2019-10-06T21:03:17Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
time="2019-10-06T21:03:17Z" level=debug msg="Creating middleware" routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd middlewareName=pipelining middlewareType=Pipelining serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 entryPointName=websecure
time="2019-10-06T21:03:17Z" level=debug msg="Creating load-balancer" serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd
time="2019-10-06T21:03:17Z" level=debug msg="Creating server 0 http://10.40.0.6:8080" serverName=0 entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3
time="2019-10-06T21:03:17Z" level=debug msg="Added outgoing tracing middleware default-traefikingressroute-8a41b49845d7fa72a5b3" middlewareName=tracing middlewareType=TracingForwarder entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd
time="2019-10-06T21:03:17Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2019-10-06T21:03:17Z" level=debug msg="Try to challenge certificate for domain [traefik.hostname.com] founded in HostSNI rule" rule="Host(`traefik.hostname.com`)" providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3
time="2019-10-06T21:03:17Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.hostname.com\"]..." rule="Host(`traefik.hostname.com`)" providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3
time="2019-10-06T21:03:17Z" level=debug msg="No ACME certificate generation required for domains [\"traefik.hostname.com\"]." providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)"
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" servicePort=9001 providerName=kubernetescrd ingress=web namespace=default serviceName=whoami-service
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" servicePort=9001 providerName=kubernetescrd ingress=websecure namespace=default serviceName=whoami-service
time="2019-10-06T21:03:19Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" ingress=websecure namespace=default providerName=kubernetescrd serviceName=whoami-service servicePort=9001
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" servicePort=9001 namespace=default serviceName=whoami-service providerName=kubernetescrd ingress=web
time="2019-10-06T21:03:19Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" servicePort=9001 ingress=websecure namespace=default providerName=kubernetescrd serviceName=whoami-service
time="2019-10-06T21:03:19Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=error msg="Cannot create service: subset not found" serviceName=whoami-service servicePort=9001 providerName=kubernetescrd ingress=web namespace=default
time="2019-10-06T21:03:19Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:19Z" level=debug msg="Adding certificate for domain(s) traefik.hostname.com"
time="2019-10-06T21:03:19Z" level=debug msg="Adding certificate for domain(s) whoami.hostname.com"
time="2019-10-06T21:03:19Z" level=debug msg="No default certificate, generating one"
time="2019-10-06T21:03:19Z" level=error msg="the service \"default-web-0143cec8478ccfc49ec6@kubernetescrd\" does not exist" entryPointName=web routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd
time="2019-10-06T21:03:19Z" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2019-10-06T21:03:19Z" level=error msg="the service \"default-websecure-0143cec8478ccfc49ec6@kubernetescrd\" does not exist" entryPointName=websecure routerName=default-websecure-0143cec8478ccfc49ec6@kubernetescrd
time="2019-10-06T21:03:19Z" level=debug msg="Creating middleware" entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 middlewareName=pipelining middlewareType=Pipelining
time="2019-10-06T21:03:19Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3
time="2019-10-06T21:03:19Z" level=debug msg="Creating server 0 http://10.40.0.6:8080" entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 serverName=0
time="2019-10-06T21:03:19Z" level=debug msg="Added outgoing tracing middleware default-traefikingressroute-8a41b49845d7fa72a5b3" entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd middlewareType=TracingForwarder middlewareName=tracing
time="2019-10-06T21:03:19Z" level=debug msg="Creating middleware" entryPointName=websecure middlewareName=traefik-internal-recovery middlewareType=Recovery
time="2019-10-06T21:03:19Z" level=debug msg="Try to challenge certificate for domain [traefik.hostname.com] founded in HostSNI rule" routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)" providerName=default.acme
time="2019-10-06T21:03:19Z" level=debug msg="Try to challenge certificate for domain [whoami.hostname.com] founded in HostSNI rule" providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6 rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)"
time="2019-10-06T21:03:19Z" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.hostname.com\"]..." rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)" providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6
time="2019-10-06T21:03:19Z" level=debug msg="No ACME certificate generation required for domains [\"whoami.hostname.com\"]." rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)" providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6
time="2019-10-06T21:03:19Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.hostname.com\"]..." providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)"
time="2019-10-06T21:03:19Z" level=debug msg="No ACME certificate generation required for domains [\"traefik.hostname.com\"]." routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)" providerName=default.acme
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" providerName=kubernetescrd ingress=web namespace=default serviceName=whoami-service servicePort=9001
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" servicePort=9001 ingress=websecure namespace=default serviceName=whoami-service providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" providerName=kubernetescrd ingress=web servicePort=9001 namespace=default serviceName=whoami-service
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" providerName=kubernetescrd ingress=websecure namespace=default serviceName=whoami-service servicePort=9001
time="2019-10-06T21:03:21Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" providerName=kubernetescrd ingress=web servicePort=9001 namespace=default serviceName=whoami-service
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=error msg="Cannot create service: subset not found" serviceName=whoami-service namespace=default providerName=kubernetescrd servicePort=9001 ingress=websecure
time="2019-10-06T21:03:21Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:21Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:22Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:22Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:22Z" level=debug msg="Configuration received from provider kubernetescrd: {\"http\":{\"routers\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"entryPoints\":[\"websecure\"],\"service\":\"default-traefikingressroute-8a41b49845d7fa72a5b3\",\"rule\":\"Host(`traefik.hostname.com`)\",\"tls\":{\"certResolver\":\"default\"}},\"default-web-0143cec8478ccfc49ec6\":{\"entryPoints\":[\"web\"],\"service\":\"default-web-0143cec8478ccfc49ec6\",\"rule\":\"Host(`whoami.hostname.com`) \\u0026\\u0026 PathPrefix(`/`)\",\"priority\":1},\"default-websecure-0143cec8478ccfc49ec6\":{\"entryPoints\":[\"websecure\"],\"service\":\"default-websecure-0143cec8478ccfc49ec6\",\"rule\":\"Host(`whoami.hostname.com`) \\u0026\\u0026 PathPrefix(`/`)\",\"priority\":1,\"tls\":{\"certResolver\":\"default\"}}},\"services\":{\"default-traefikingressroute-8a41b49845d7fa72a5b3\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.40.0.6:8080\"}],\"passHostHeader\":true}},\"default-web-0143cec8478ccfc49ec6\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.39.0.7:80\"}],\"passHostHeader\":true}},\"default-websecure-0143cec8478ccfc49ec6\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.39.0.7:80\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"tls\":{}}" providerName=kubernetescrd
time="2019-10-06T21:03:22Z" level=debug msg="Adding certificate for domain(s) traefik.hostname.com"
time="2019-10-06T21:03:22Z" level=debug msg="Adding certificate for domain(s) whoami.hostname.com"
time="2019-10-06T21:03:22Z" level=debug msg="No default certificate, generating one"
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Creating middleware" entryPointName=web routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-web-0143cec8478ccfc49ec6 middlewareName=pipelining middlewareType=Pipelining
time="2019-10-06T21:03:23Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-web-0143cec8478ccfc49ec6
time="2019-10-06T21:03:23Z" level=debug msg="Creating server 0 http://10.39.0.7:80" entryPointName=web routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-web-0143cec8478ccfc49ec6 serverName=0
time="2019-10-06T21:03:23Z" level=debug msg="Added outgoing tracing middleware default-web-0143cec8478ccfc49ec6" routerName=default-web-0143cec8478ccfc49ec6@kubernetescrd middlewareName=tracing middlewareType=TracingForwarder entryPointName=web
time="2019-10-06T21:03:23Z" level=debug msg="Creating middleware" entryPointName=web middlewareType=Recovery middlewareName=traefik-internal-recovery
time="2019-10-06T21:03:23Z" level=debug msg="Creating middleware" routerName=default-websecure-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-websecure-0143cec8478ccfc49ec6 middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure
time="2019-10-06T21:03:23Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=default-websecure-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-websecure-0143cec8478ccfc49ec6
time="2019-10-06T21:03:23Z" level=debug msg="Creating server 0 http://10.39.0.7:80" entryPointName=websecure serverName=0 routerName=default-websecure-0143cec8478ccfc49ec6@kubernetescrd serviceName=default-websecure-0143cec8478ccfc49ec6
time="2019-10-06T21:03:23Z" level=debug msg="Added outgoing tracing middleware default-websecure-0143cec8478ccfc49ec6" entryPointName=websecure routerName=default-websecure-0143cec8478ccfc49ec6@kubernetescrd middlewareName=tracing middlewareType=TracingForwarder
time="2019-10-06T21:03:23Z" level=debug msg="Creating middleware" middlewareName=pipelining routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd entryPointName=websecure serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 middlewareType=Pipelining
time="2019-10-06T21:03:23Z" level=debug msg="Creating load-balancer" serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd entryPointName=websecure
time="2019-10-06T21:03:23Z" level=debug msg="Creating server 0 http://10.40.0.6:8080" serverName=0 entryPointName=websecure serviceName=default-traefikingressroute-8a41b49845d7fa72a5b3 routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Added outgoing tracing middleware default-traefikingressroute-8a41b49845d7fa72a5b3" middlewareName=tracing middlewareType=TracingForwarder entryPointName=websecure routerName=default-traefikingressroute-8a41b49845d7fa72a5b3@kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=websecure
time="2019-10-06T21:03:23Z" level=debug msg="Try to challenge certificate for domain [traefik.hostname.com] founded in HostSNI rule" providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)"
time="2019-10-06T21:03:23Z" level=debug msg="Try to challenge certificate for domain [whoami.hostname.com] founded in HostSNI rule" providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6 rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)"
time="2019-10-06T21:03:23Z" level=debug msg="Looking for provided certificate(s) to validate [\"traefik.hostname.com\"]..." providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)"
time="2019-10-06T21:03:23Z" level=debug msg="No ACME certificate generation required for domains [\"traefik.hostname.com\"]." providerName=default.acme routerName=default-traefikingressroute-8a41b49845d7fa72a5b3 rule="Host(`traefik.hostname.com`)"
time="2019-10-06T21:03:23Z" level=debug msg="Looking for provided certificate(s) to validate [\"whoami.hostname.com\"]..." providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6 rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)"
time="2019-10-06T21:03:23Z" level=debug msg="No ACME certificate generation required for domains [\"whoami.hostname.com\"]." providerName=default.acme routerName=default-websecure-0143cec8478ccfc49ec6 rule="Host(`whoami.hostname.com`) && PathPrefix(`/`)"
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:23Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="No secret name provided" providerName=kubernetescrd
time="2019-10-06T21:03:25Z" level=debug msg="Skipping Kubernetes event kind *v1.Endpoints" providerName=kubernetescrd
1 Like
2

Did you find a solution to this? I'm also having this same issue. The CRD is correctly discovering services but yet I still receive the error:

the service "whoami-{random_string}@kubernetescrd" does not exist

UI Example:

VV7C8
VV7C8.png488×583 27.2 KB

1 Like
3

Same problem here, running k8s in AWS.
What a pity!

4

Just for testing purposes try to deploy everything into the same namespace.

5

I've seen this when deployed entirely to 'default'.

6

Guys, did you get any solution to the issue, I am also facing the same on bare metal k8s cluster

7

I ran into a similar issue with middleware resources not being discovered and the problem was that the resources needs the namespace and a hyphen pre-pended at the beginning, even if everything was all in the same namespace.

E.g. in the default namespace, I had to reference my middleware as

"default-mymiddleware@kubernetescrd"