Unable to reach the service. getting 404

I'm trying to use Traefik as an Ingress-Controller for my kubernetes cluster. I'm using IBM Kubernetes cluster. I'm facing couple of issues.

  • I'm unable to reach the service when trying to access it via browser/curl. Getting error Host not found.

  • I'm trying to get a ssl certificate using LetsEncrypt tls-alpn challenge.
    Error

    level=error msg="Unable to obtain ACME certificate for domains "demo.example.in": unable to generate a certificate for the domains [demo.example.in]: error: one or more domains had a problem:\n[demo.example.in] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Cannot negotiate ALPN protocol "acme-tls/1" for tls-alpn-01 challenge, url: \n" providerName=letsencrypt.acme routerName=jitsi-traefik-ingress-23a7c4c2c5a70da0e821@kubernetescrd rule="Host(demo.example.in)"

FLOW
I have a service web with a namespace: jitsi which is running on port 80. I'm trying to use Traefik for handling https and then forwarding it to the service web on port 80.

PS: I'm already using Traefik with my docker-swarm cluster and It's running fine. I'm new to Kubernetes and there's a lot of difference how we configure Traefik for docker-swarm and Kubernetes.

For configuring Traefik I'm following official link.

Configuration

Ingress Definition

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutes.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRoute
    plural: ingressroutes
    singular: ingressroute
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: middlewares.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: Middleware
    plural: middlewares
    singular: middleware
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressroutetcps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteTCP
    plural: ingressroutetcps
    singular: ingressroutetcp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ingressrouteudps.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: IngressRouteUDP
    plural: ingressrouteudps
    singular: ingressrouteudp
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsoptions.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSOption
    plural: tlsoptions
    singular: tlsoption
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tlsstores.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TLSStore
    plural: tlsstores
    singular: tlsstore
  scope: Namespaced

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: traefikservices.traefik.containo.us

spec:
  group: traefik.containo.us
  version: v1alpha1
  names:
    kind: TraefikService
    plural: traefikservices
    singular: traefikservice
  scope: Namespaced

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - traefik.containo.us
    resources:
      - middlewares
      - ingressroutes
      - traefikservices
      - ingressroutetcps
      - ingressrouteudps
      - tlsoptions
      - tlsstores
    verbs:
      - get
      - list
      - watch

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller

roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
  - kind: ServiceAccount
    name: traefik-ingress-controller
    namespace: default

Service

apiVersion: v1
kind: Service
metadata:
  name: traefik

spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
    - protocol: TCP
      name: admin
      port: 8080
    - protocol: TCP
      name: websecure
      port: 443
  selector:
    app: traefik

Service-web.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    service: web
  name: web
  namespace: jitsi
spec:
  ports: 
  - name: web
    port: 80
    targetPort: 80
  selector:
    k8s-app: web

Deployment-traefik.yaml

## create a new ServiceAccount to provide Traefik with the identity in your cluster. ##
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: traefik-ingress-controller

---
### Deploy Traefik to a Cluster ###
## We can use Deployment, DaemonSet or Helm Chart
kind: Deployment
apiVersion: apps/v1
metadata:
  namespace: default
  name: traefik
  labels:
    app: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik
  template:
    metadata:
      labels:
        app: traefik
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - name: traefik
        image: traefik:2.2
        imagePullPolicy: IfNotPresent
        resources:
            limits:
              memory: 400Mi
              cpu: 400m
            requests:
              memory: 400Mi
              cpu: 400m
        args:
        - --log=true
        - --log.level=DEBUG
        - --accesslog
        - --providers.kubernetescrd
        #- --providers.kubernetesingress=true
        - --entryPoints.web.address=:80
        - --entrypoints.web.http.redirections.entryPoint.to=websecure
        - --entrypoints.web.http.redirections.entryPoint.scheme=https
        - --entryPoints.websecure.address=:443
        - --certificatesResolvers.letsencrypt.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory
        - --certificatesResolvers.letsencrypt.acme.tlsChallenge
        - --certificatesresolvers.letsencrypt.acme.email=myemail@gmail.com
        - --certificatesResolvers.letsencrypt.acme.storage=/data/acme.json        
        ports:
        - name: web
          containerPort: 80
        - name: admin
          containerPort: 8080
        - name: websecure
          containerPort: 443  
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        volumeMounts:
          - mountPath: /data
            name: storage-volume    

      restartPolicy: Always
      volumes:
        - name: storage-volume
          persistentVolumeClaim:
              claimName: traefik-acme-storage

deployment-web.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: web
  name: web
  namespace: jitsi
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      k8s-app: web
  template:
    metadata:
      labels:
        k8s-app: web
    spec:
      volumes:
        - name: web
          configMap:
              name: web
              items:
                - key: welcomePageAdditionalContent.html
                  path: welcomePageAdditionalContent.html
                - key: plugin.head.html
                  path: plugin.head.html
                - key: config.js
                  path: config.js
                - key: interface_config.js
                  path: interface_config.js 
      containers:
        - name: web
          image: jitsi/web
          imagePullPolicy: Always
          readinessProbe:
              httpGet:
                port: 80
          ports:
            - name: web
              containerPort: 80
          resources:
            limits:
              memory: 300Mi
              cpu: 400m
            requests:
              memory: 300Mi
              cpu: 400m
          volumeMounts:
          - name: web
            mountPath: /usr/share/jitsi-meet/static/welcomePageAdditionalContent.html
            subPath: welcomePageAdditionalContent.html
          - name: web
            mountPath: /usr/share/jitsi-meet/plugin.head.html
            subPath: plugin.head.html
          - name: web
            mountPath: /defaults/config.js
            subPath: config.js
          - name: web
            mountPath: /defaults/interface_config.js
            subPath: interface_config.js

          env:
            - name: ENABLE_AUTH
              value: "1"
            - name: ENABLE_GUESTS
              value: "1"
            - name: ENABLE_RECORDING
              value: "0"
            - name: DISABLE_HTTPS
              value: "1"
            - name: PUBLIC_URL
              value: http://demo.example.in
            - name: HTTP_PORT
              value: "80"
            - name: XMPP_SERVER
              value: prosody
            - name: JICOFO_AUTH_USER
              value: focus
            - name: XMPP_DOMAIN
              value: demo.example.in
            - name: XMPP_AUTH_DOMAIN
              value: auth.demo.example.in
            - name: XMPP_INTERNAL_MUC_DOMAIN
              value: internal-muc.demo.example.in
            - name: XMPP_BOSH_URL_BASE
              value: http://prosody:5280
            - name: XMPP_MUC_DOMAIN
              value: muc.demo.example.in
            - name: TZ
              value: Asia/Kolkata
            - name: JVB_TCP_HARVESTER_DISABLED
              value: "true"
            - name: JIBRI_BREWERY_MUC
              value: jibribrewery
            - name: JIBRI_PENDING_TIMEOUT
              value: "90"
            - name: JIBRI_XMPP_USER
              value: jibri
            - name: JIBRI_RECORDER_USER
              value: recorder
            - name: JIBRI_XMPP_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: jitsi-config
                  key: JIBRI_XMPP_PASSWORD
            - name: JIBRI_RECORDER_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: jitsi-config
                  key: JIBRI_RECORDER_PASSWORD

Traefik Routers

ingress-traefik.yaml

## This need to seperated as It was causing some issue with IBM k8's cluster
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-ingress
  namespace: jitsi
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`demo.example.in`)
    kind: Rule
    services:
    - name: web
      port: 80
  tls:
    certResolver: letsencrypt

Please help.