Properly annotating Ingress to serve gRPC

Traefik Traefik v1
1

We've setup traefik with helm chart version 1.61.0 (because automatic encryption wasn't working with the newest version - see https://github.com/containous/traefik/issues/4850). That's a bit beside the point.

ssl:
  enabled: true
  enforced: false

We've been able to get Let's Encrypt working, and properly access via https and server some html. Cool.

I'm now trying to properly annotate a gRPC service's ingress so that I can successfully request... my data with a python gRPC client. (I've port forwarded to the pod and made the same client request and it servers properly so I know it's something with my traefik configuration...)

This is what my ingress looks like now:

kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/protocol: h2c
    ingress.kubernetes.io/ssl-passthrough: "true"
    kubernetes.io/ingress.class: traefik
    traefik.protocol: h2c
  creationTimestamp: "2019-07-04T08:59:33Z"
  generation: 2
  name: dataservice
  namespace: <my namespace>
  resourceVersion: "21175139"
  selfLink: /apis/extensions/v1beta1/namespaces/<my namespace>/ingresses/dataservice
  uid: 0aa6f4ee-9e3a-11e9-9bb9-9e2c1485c626
spec:
  rules:
  - host: dataservice.api.<my domain>.io
    http:
      paths:
      - backend:
          serviceName: dataservice
          servicePort: 80
        path: /
status:
  loadBalancer: {}

I get a redirect from my request like this... i'm trying to figure out whether I need to add another specific header to my client request or if i've got the wrong idea with my ingress annotations etc etc...

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
        status = StatusCode.CANCELLED
        details = "Received http2 header with status: 308"

This might be relevant, I just need traefik to forward the request it get's to the backend, but it doesn't appear to work that way...

2

Hello,

The annotation traefik.protocol: h2c is not a valid annotation, the right annotation is ingress.kubernetes.io/protocol: h2c

ingress.kubernetes.io/ssl-passthrough: "true" is not a supported Traefik annotation.

https://docs.traefik.io/configuration/backends/kubernetes/#annotations

Could give the Traefik log?

1 Like
3

We've moved along quite a bit, in a few moments i'll share the current set of configuration and also the logs. Thanks!

The basic issue is that we've got a properly working frontend, running https, but when we send requests with our gRPC client to the frontend, it doesn't seem to get through to our gRPC backend...

4

FYI, the h2c protocol is made for http/2 without TLS.

5

So once we properly annotated our backend service with h2c, AND AND AND (importantly) fed the proper front end certificate into our python gRPC client then we got through! It'll take a few more cycles but i'll come back with more details.

6

Ok so here's how we got it to work:

On our gRPC client we had to include the certificate from the traefik served frontend (https secured by acme).

This was the important second argument to grpc.secure_channel --> grpc.ssl_channel_credentials()

import os
import grpc
import echo_pb2
import echo_pb2_grpc
import requests
import time
# import ssl

print('Connecting...')
domain = 'test.api.domain.io'
protected_url = f"https://{domain}/"

channel = grpc.secure_channel(f'{domain}:443', grpc.ssl_channel_credentials())
stub = echo_pb2_grpc.EchoServiceStub(channel)

### build up authentication token here...

res = stub.Echo(echo_pb2.envelope(message="Test 1 2 3 Test"), metadata=[token])

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: {{ .Values.applicationservicename }}
  annotaions:
  annotations:
    kubernetes.io/ingress.class: traefik
    ingress.kubernetes.io/protocol: h2c
    ingress.kubernetes.io/auth-response-headers: X-Userinfo, X-Id-Token, X-Access-Token, Authorization
    ingress.kubernetes.io/auth-type: forward
    ingress.kubernetes.io/auth-url: https://authentication.api.middleware.io/verify?fallback_plugin=0&config_token={{ .Values.ingress.authentication.config_token }}
spec:
  tls:
  - hosts: 
    - {{ .Values.ingress.host }}
  rules:
  - host: {{ .Values.ingress.host }}
    http:
      paths:
      - backend:
          serviceName: {{ .Values.applicationservicename }}
          servicePort: 80
        path: /