Getting started with Kubernetes Gateway API and Traefik

We're continuing our in-depth series on Traefik 3. If you missed it, be sure to read the previous articles on migrating from Traefik v2, WASM support with Coraza WAF, Open Telemetry, and SPIFFE, Tailscale, and HTTP/3. This article dives into how to get started with GatewayAPI and Traefik.

In 2015, when Traefik was just born, Kubernetes released the first version of its Ingress specifications. The goal of Ingress was to provide a straightforward, vendor-neutral method for exposing Kubernetes services. However, the premise has quickly been broken.

While Ingress could handle basic service exposure, adding functionalities like rate-limiting or IP checking required numerous vendor-specific annotations.

To address these limitations, a new standard began to take shape in 2019: GatewayAPI was born. Officially released in its v1.0 form at the end of 2023, the GatewayAPI aims to be the new standard for exposing services in Kubernetes. It offers a set of core rules designed to meet most needs while allowing the Gateway Controller to extend these rules with their features.

Traefik has been a Kubernetes first-class citizen for a long time, initially supporting Ingress and later expanding with our IngressRoutes to surpass Ingress limitations, adding features like TCP, UDP, and structured options declaration.

We quickly embraced the big changes in Kubernetes with the Gateway API, offering early experimental support since 2022.

With the release of Traefik v3.1, we’ve taken a major step forward: our GatewayAPI provider is now production-ready. Traefik v3.1 today meets and exceeds 100% of the core requirements, as shown in our SIG network conformance tests report.

Let’s see together how you can start with GatewayAPI and Traefik v3!

Install Traefik as GatewayController

To begin, ensure you have installed the most recent version of Traefik on your Kubernetes cluster, utilizing the Helm Chart for streamlined deployment. This setup enables the GatewayAPI provider to effectively discover and expose your applications.

Define your installation configuration

First, customize your values to enable Traefik to discover GatewayAPI objects on your Kubernetes cluster. Create a values.yaml file with the following minimal configuration:

## File values.yaml ##
providers:
  # Disable the Ingress provider (optional)
  # We do not want to use Ingress objects anymore!
  kubernetesIngress:
	enabled: false
  # Enable the GatewayAPI provider
  kubernetesGateway:
	enabled: true
# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  namespacePolicy: All

Next, execute the following commands to deploy Traefik in the traefik namespace using the previously described configuration:

$ helm repo add traefik https://traefik.github.io/charts
$ helm repo update
$ kubectl create namespace traefik
$ helm upgrade --install --version 0.29.1 --namespace traefik traefik traefik/traefik -f values.yaml

And voilà! You have deployed Traefik on your Kubernetes cluster:

$ kubectl describe deployments.apps traefik --namespace traefik
Name:               	traefik
Namespace:          	traefik
…
  Containers:
   traefik:
	Image:   	docker.io/traefik:v3.1
	Ports:   	9100/TCP, 9000/TCP, 8000/TCP, 8443/TCP
	Host Ports:  0/TCP, 0/TCP, 0/TCP, 0/TCP
	Args:
  	--entryPoints.web.address=:8000/tcp
  	--entryPoints.websecure.address=:8443/tcp
  	--api.dashboard=true
  	--ping=true
  	--providers.kubernetescrd
  	--providers.kubernetesgateway
  	--entryPoints.websecure.http.tls=true
	…

As you can see, both kubernetescrd and kubernetesgateway providers are enabled. The kubernetescrd provider allows you to define Traefik specific resources like Middlewares.

The entrypoints web and websecure will be used to expose your applications.

Additionally, when Traefik is installed with the GatewayAPI provider enabled, it automatically creates a default GatewayClass named traefik:

$ kubectl describe GatewayClass traefik
Name:     	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	GatewayClass
…
  Controller Name:  traefik.io/gateway-controller

As you can see above, the option Controller Name is set to traefik.io/gateway-controller. Thus, it allows you to expose every Gateway attached to the traefik GatewayClass using your Traefik Gateway Controller.

Additionally, the Gateway traefik-gateway is also deployed:

$ kubectl describe Gateway traefik --namespace traefik
Name:     	traefik-gateway
Namespace:	traefik
…
API Version:  gateway.networking.k8s.io/v1
Kind:     	Gateway
…
Spec:
  Gateway Class Name:  traefik
  Listeners:
	Allowed Routes:
  	  Namespaces:
    	    From:  All
	Name:  	web
	Port:  	8000
	Protocol:  HTTP

This Gateway allows you to expose HTTPRoute in HTTP on the port 8000 of your Traefik Gateway Controller.

Let’s now expose your first application using a GatewayAPI HTTPRoute.

Deploy your first HTTPRoute

A GatewayAPI HTTPRoute enables you to expose an application through Gateways using HTTP(S).

To do this, reference the Services to reach (using the backendRefs option) and the Gateways (using the parentRef option) as shown in the snippet below:

# Application to expose
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
    app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - name: whoami
        image: traefik/whoami
---
# Service to reach the application on the cluster
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
  - port: 80
      name: whoami
  selector:
    app: whoami
---
# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
	- path:
    	    type: PathPrefix
    	    value: /
	backendRefs:
	- name: whoami
  	  namespace: whoami
  	  port: 80

This example demonstrates how to expose a web server that displays some headers when Traefik is accessed using the hostname whoami.myexample.io. You can use the following curl command to verify that the application is correctly exposed:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-7nqmf
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.9
IP: fe80::e8c0:86ff:feba:5e06
RemoteAddr: 10.42.0.14:59316
GET / HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Real-Ip: 10.42.0.1

Congratulations, you've successfully deployed your first HTTPRoute with Traefik!

Now, let's take it a step further by adding a few operations to the requests.

Go beyond with Filters

GatewayAPI Filters enable Traefik to perform various operations on requests and responses.

There are three types of filters:

  1. Core Filters: Mandatory filters for every GatewayController, such as requestHeaderModifier and requestRedirect.
  2. Extended Filters: Optional filters for GatewayControllers, such as responseHeaderModifier and requestMirror.
  3. ExtensionRef Filters: Additional filters provided by the GatewayController. In Traefik, these are the HTTP middlewares.

Let's modify the HTTPRoute to: Add the header x-post-topic using the core filter. Add the prefix gatewayapi to the request using the middleware AddPrefix.

# HTTPRoute with the Filters
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
  - name: traefik-gateway
    namespace: traefik
  hostnames:
  - whoami.myexample.io
  rules:
  - matches:
    - path:
        type: PathPrefix
    	value: /
    backendRefs:
    - name: whoami
      namespace: whoami
      port: 80
    filters:
    # Core filter which adds a header
    - type: RequestHeaderModifier
      requestHeaderModifier:
        add:
          - name: x-post-topic
            value: GatewayAPI
    # ExtensionRef filter to use the Traefik Middleware AddPrefix
    - type: ExtensionRef
      extensionRef:
        group: traefik.io
        kind: Middleware
        name: addprefix
---
# Traefik Middleware AddPrefix
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: addprefix
  namespace: whoami
spec:
  addPrefix:
    prefix: /gatewayapi

You can check the modification using a curl command:

$ curl http://whoami.myexample.io/
Hostname: whoami-697f8c6cbc-2xkpr
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.10
IP: fe80::b8b5:6fff:fe48:19e9
RemoteAddr: 10.42.0.14:40044
GET /gatewayapi/ HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Post-Topic: GatewayAPI
X-Real-Ip: 10.42.0.1

As we have seen, the GatewayAPI integration in Traefik simplifies the initial setup significantly. We successfully exposed our first HTTPRoute and added a couple of operations to the requests. Traefik has many more GatewayAPI capabilities, such as TCPRoute, TLSRoute, and ReferenceGrant that are not covered in this blog post, but we highly encourage you to explore them further.

Conclusion

GatewayAPI is undoubtedly the new standard for exposing your resources in a Kubernetes cluster. Its specifications already surpass those of Ingress, and with a strong community backing it, GatewayAPI will continue to grow.

As a Kubernetes first-class citizen, Traefik is of course strongly supporting this effort and will continue to contribute to this community project.

We've improved status management, updated route priority rules, and introduced the ReferenceGrant feature to make Traefik even more robust and flexible for your Kubernetes setup.

But it’s just the beginning! We plan to further advance this integration by adding support for features like GRPCRoute, UDPRoute, and BackendTLSPolicy.

Our goal is to provide comprehensive support, enabling you to fully leverage GatewayAPI in your Kubernetes journey. Stay tuned for more updates!

Useful Links


This is a companion discussion topic for the original entry at https://traefik.io/blog/getting-started-with-kubernetes-gateway-api-and-traefik

There is a mistake in manifest you mentioned. I struggled a lot to find the issue.

the parameter "namespacePolicy" is not under gateway it must be under "gateway -> listeners". After changing that to:

# Allow the Gateway to expose HTTPRoute from all namespaces
gateway:
  listeners:
    web:
      namespacePolicy: All
    websecure:
      namespacePolicy: All

the deployment of the whoami app were successfully over the traefik proxy available.

Unfortuantely it's not working with V30.0.1 nor V30.

Error: UPGRADE FAILED: failed to create resource: Gateway.gateway.networking.k8s.io "traefik-gateway" is invalid: spec.listeners: Invalid value: "array": tls must be specified for protocols ['HTTPS', 'TLS']

link to github issue

@john222 what version are you using?

@john222 I wasn't able to get things working as expected with/without the updated code that you provided in your post. At this time, I'm getting the following error message:

➜ curl http://whoami.myexample.io
curl: (6) Could not resolve host: whoami.myexample.io

Here are the issues that I experienced with the post:

Finally, I'm using the following within my local development environment:

  • Traefik v3.1
  • Traefik Chart v30.0.2
  • macOS 14.6
  • K8s v1.30.3
  • Minikube v1.33.1
  • OrbStack v1.6.4
  • MetalLB v0.14.8

After installing the missing CRDs, I was able to get past this issue. For example, I executed the following line prior to installing Traefik:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/experimental-install.yaml

Hello @rohanrehman

this is because the idents are wrong. You have to correct the idents to make it working.

Hello @conradwt

Of course this is not working when the dns record whoami.myexample.io does not exist. You have to change it to one of your dns records to get it working. They just provided this dns name as example. The dns record have to point to the load balancer ip your metallb have assigned to traefik.

Ah yes, you are right, i also had to install gateway api cdr's to get it working.

Thanks John my issue was already solved last night and was about to update this thread.
as @conradwt mentioned it was the missing CRDs.

@nicomengin I strongly suggest an update that blog post and add the following
requirements as listed here in the docs but not in the blog post.

1. Install Gateway API CRDs from the Experimental channel.
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/experimental-install.yaml

2. Install/update the Traefik [RBAC]

kubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v3.1/docs/content/reference/dynamic-configuration/kubernetes-gateway-rbac.yml

Also as John mentioned I cannot enough express the importance of correct yaml indentation when posting yaml snippets.

When you post any code in a getting started blog post, it should work without any modification or at least document what the user must change to get things working.

1 Like

I was able to get things working after doing the following:

  1. installing the Traefik RBAC
  2. creating a namespace
kubectl create namespace whoami
  1. commenting out hostnames section and applying whoami application YAML to the cluster
# Application to expose
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  namespace: whoami
spec:
  replicas: 3
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami
---
# Service to reach the application on the cluster
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: whoami
  labels:
    app: whoami
spec:
  type: ClusterIP
  ports:
    - port: 80
      name: whoami
  selector:
    app: whoami
---
# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
    - name: traefik-gateway
      namespace: traefik
  # hostnames:
  #   - whoami.myexample.io
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: whoami
          namespace: whoami
          port: 80
1 Like

No need to comment out the hostnames it's quite important to know how to use them if you are routing based on url.

Just indent hostnames correctly. it should be under the same column as parentRefs

  parentRefs:
    - name: traefik-gateway
      namespace: traefik
  hostnames:
    - whoami.myexample.io

Yes, the hostnames section is properly formatted and it's not a formatting issue here. It's because there's no DNS entry for whoami.myexample.io. Thus, I'm getting the following error message:

➜ curl whoami.myexample.io           
curl: (6) Could not resolve host: whoami.myexample.io

From the blog post, the output should be similar to the following:

Hostname: whoami-697f8c6cbc-7nqmf
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.9
IP: fe80::e8c0:86ff:feba:5e06
RemoteAddr: 10.42.0.14:59316
GET / HTTP/1.1
Host: whoami.myexample.io
User-Agent: curl/7.88.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.0.1
X-Forwarded-Host: whoami.myexample.io
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-5d476f955f-cn7xs
X-Real-Ip: 10.42.0.1

Correct?

Hi, I'm super new to Kubernetes and trying to follow this tutorial as well after being unable to set up TLS communication with a custom CA between Traefik 2 and my app pods. I thought I would give the new Gateway API method a try, since the BackendTLSPolicy feature seems to be exactly what I am looking for.

I have set up a k3s cluster with 3 nodes on Turing Pi board on my local network, with --disable=traefik specified. I then followed the steps in the blog post (and comments) to deploy Traefik 3.1.0 with Helm, and completed the whoami deployment with hostnames: gateway.local. This all worked fine (except for the horrific yaml indentation, thanks @conradwt for posting a working example).

My cluster controller has the IP 10.0.0.10 on the local network, and I defined 10.0.0.10 gateway.local in my /etc/hosts file. When I import the cluster in a cloud-hosted Rancher instance, I can see all the pods created in traefik and whoami namespaces respectively.

Browsing to http://gateway.local, http://gateway.local/ https://gateway.local or https://gateway.local/ all fails with 404 page not found. The Traefik container is showing 2024-08-01T12:27:29Z ERR Unable to create Gateway status error="1 error occurred:\n\t* No TLS configuration for Gateway Listener websecure:8443 and protocol \"HTTPS\"\n\n" gateway=traefik-gateway namespace=traefik providerName=kubernetesgateway but these logs seem unrelated to my attempts to access whomai. I'm also unable to access the dashboard under /dashboard. How can I try and debug this to get it working?

1 Like

I was finally able to access the dashboard by editing the Traefik deployment and adding --api.insecure to spec.template.spec.containers[args]. Would be great if there some logs visible anywhere indicating that this was the issue all along! Why is it necessary to set this to insecure?

I'm still unable to access the whoami service as described in the blog post, and again there are no logs to help me understand why, it's just a black box to someone new like me.

Sorry for the later reply...Traefik's docs has a "particular" style, so I can understand the need for clarification.
commenting out the hostnames will stop the HTTPRoute from doing a "matching" check.

But the point I was making was that not leveraging HTTPRoute fully will defeat the whole purpose of using Traefik.

When you (...as in generally anyone) end up build web apps that are containerized ... For example 10 react apps that you serve a corp website or a sass app that you want reachable through the browser, you are limited a few ports ( 80, 443 or 8443 )
Those containers will all expose different NodePorts within your cluster that will still have traffic directed from Traefik's 80,443,8443.

Those multiple domains that you want to display/serve the corresponding container will need to leverage HTTPRoute hostname, as the DNS setup for each of those domains are all going to be posting to the same few ports (80, 443, 8443).

The hostnames is what can differentiate those requests.

What we do is route our AI services to a different cluster that have GPU's to run the AI workloads (container's themselves have access to the GPU), and for in memory services such a redis which hold everything in ram, another cluster for the Redis cluster itself, with larger ram specs.
We even have a dedicated cluster for QR code generation.

My advice on using Traefik or any other API gateway is experiment with the hostnames with free dns providers / services.

Reduce as much complexity as possible.
Online posts and tutorials have a shelf life of 2 months, there are more a marketing effort.

Start with small steps and a few open ports and more importantly move away from testing within your local network and expose external ports attached to a real domain.

you are on the right track with using Helm charts, just redo your steps geared towards testing with an external url.

On Traefik's annotations... the direction of V3 and the Gateway Api is to move away form those annotations and having simpler yaml files.

No worries about the late reply and I appreciate your expertise on the subject matter. Anyway, I'll keep experimenting with the K8s Gateway API HTTPRoute hostnames, rules, and filters.

Thanks, I'm back at trying this and just spent another 5 hours trying to configure an HTTPRoute to direct traffic to my whoami deployment. I take your point that it makes sense to use this on a real domain, but I need to prove to my manager that the entire concept of using Kubernetes/Traefik as a routing tool is workable on our local network first. I'm familiar with nginx reverse proxies and this is theoretically not a difficult task, particularly because nginx logs are generally quite helpful. All I get from Traefik is 404 page not found, and I'm not even sure if this is coming from Traefik or some Kubernetes component because no logs are written by the Traefik pod.

I have created a deployment, service and httproute exactly as shown by @conradwt in the post above with corrected indentation. I also tried commenting out the hostnames section, or using various PathPrefix values to try and have Traefik route traffic based on those instead, like this:

# HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: whoami-httproute
  namespace: whoami
spec:
  parentRefs:
    - name: traefik-gateway
      namespace: traefik
  # hostnames:
  #   - gateway.local
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /whoami/
      backendRefs:
        - name: whoami
          namespace: whoami
          port: 80

None of this returns anything except 404 page not found, regardless of whether I access it by IP address or a hostname like gateway.local from my laptop with this name specified in /etc/hosts. It also responds with 404 when I try to access it with curl http://localhost/whoami/ from any of the three nodes in the cluster. No logs are written anywhere.

I am still able to access the Traefik dashboard under http://127.0.0.1:9000/dashboard/ when running kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name --namespace traefik) 9000:9000 --address 0.0.0.0 --namespace traefik from my laptop. But I can't find any way to create working ingress. Would really appreciate a pointer to the right docs or a relevant forum post, thanks!