# Unable to create working HTTPRoute with Traefik 3.1.0 on K3s **URL:** \ **Category:** Traefik v3 (latest)\ **Tags:** kubernetes-ingress\ **Created:** [August 14, 2024, 3:32pm UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013 "2024-08-14T15:32:58Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![strophy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/strophy/32/9119_2.png) [@strophy](https://community.traefik.io/u/strophy)\ **Post date:** [August 14, 2024, 3:32pm UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/1 "2024-08-14T15:32:58Z")
Hi, I'm starting a separate post to the discussion of the [recent blog post](https://traefik.io/blog/getting-started-with-kubernetes-gateway-api-and-traefik/) on how to set up Traefik 3.1.0 on Kubernetes and create a simple HTTPRoute. I have a 3-node K3s cluster on my local network installed with `--disable=traefik`. The node IPs on my local network are: - 10.0.0.192 (control plane) - 10.0.0.35 - 10.0.0.80 I have copied `~/.kube/config` from the control plane to my local machine and commands like `kubectl get pods -A` show 3 pods running: ```auto $ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-576bfc4dc7-frvvd 1/1 Running 0 23m kube-system local-path-provisioner-86f46b7bf7-rtjcx 1/1 Running 0 23m kube-system metrics-server-557ff575fb-8sztf 1/1 Running 0 23m ``` So far so good. I now attempt to install Traefik 3.1.0 using the Helm chart. First we need to install experimental gateway CRDs to avoid the error `Error: UPGRADE FAILED: failed to create resource: Gateway.gateway.networking.k8s.io "traefik-gateway" is invalid: spec.listeners: Invalid value: "array": tls must be specified for protocols ['HTTPS', 'TLS']` as described [here](https://github.com/traefik/traefik-helm-chart/issues/1147#issuecomment-2257670943). ```auto kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/experimental-install.yaml ``` I then create `values.yaml` as described in the blog post: ```auto providers: kubernetesIngress: enabled: false kubernetesGateway: enabled: true gateway: listeners: web: namespacePolicy: All websecure: namespacePolicy: All ``` Next we install Traefik 3.1.0 using the Helm chart, currently v30.0.2: ```auto $ helm upgrade --install --namespace traefik traefik traefik/traefik -f values.yaml Release "traefik" has been upgraded. Happy Helming! NAME: traefik LAST DEPLOYED: Wed Aug 14 15:49:04 2024 NAMESPACE: traefik STATUS: deployed REVISION: 3 TEST SUITE: None NOTES: traefik with docker.io/traefik:v3.1.0 has been deployed successfully on traefik namespace ! ``` Let's inspect the resources it created: ```auto $ kubectl describe deployments.apps traefik --namespace traefik Name: traefik Namespace: traefik CreationTimestamp: Wed, 14 Aug 2024 15:42:31 +0200 Labels: app.kubernetes.io/instance=traefik-traefik app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=traefik helm.sh/chart=traefik-30.0.2 Annotations: deployment.kubernetes.io/revision: 1 meta.helm.sh/release-name: traefik meta.helm.sh/release-namespace: traefik Selector: app.kubernetes.io/instance=traefik-traefik,app.kubernetes.io/name=traefik Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable StrategyType: RollingUpdate MinReadySeconds: 0 RollingUpdateStrategy: 0 max unavailable, 1 max surge Pod Template: Labels: app.kubernetes.io/instance=traefik-traefik app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=traefik helm.sh/chart=traefik-30.0.2 Annotations: prometheus.io/path: /metrics prometheus.io/port: 9100 prometheus.io/scrape: true Service Account: traefik Containers: traefik: Image: docker.io/traefik:v3.1.0 Ports: 9100/TCP, 9000/TCP, 8000/TCP, 8443/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP Args: --global.checknewversion --global.sendanonymoususage --entryPoints.metrics.address=:9100/tcp --entryPoints.traefik.address=:9000/tcp --entryPoints.web.address=:8000/tcp --entryPoints.websecure.address=:8443/tcp --api.dashboard=true --ping=true --metrics.prometheus=true --metrics.prometheus.entrypoint=metrics --providers.kubernetescrd --providers.kubernetesgateway --entryPoints.websecure.http.tls=true --log.level=INFO Liveness: http-get http://:9000/ping delay=2s timeout=2s period=10s #success=1 #failure=3 Readiness: http-get http://:9000/ping delay=2s timeout=2s period=10s #success=1 #failure=1 Environment: POD_NAME: (v1:metadata.name) POD_NAMESPACE: (v1:metadata.namespace) Mounts: /data from data (rw) /tmp from tmp (rw) Volumes: data: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: tmp: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: Conditions: Type Status Reason ---- ------ ------ Available True MinimumReplicasAvailable Progressing True NewReplicaSetAvailable OldReplicaSets: NewReplicaSet: traefik-7c7587b647 (1/1 replicas created) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal ScalingReplicaSet 7m3s deployment-controller Scaled up replica set traefik-7c7587b647 to 1 $ kubectl describe GatewayClass traefik Name: traefik Namespace: Labels: app.kubernetes.io/instance=traefik-traefik app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=traefik helm.sh/chart=traefik-30.0.2 Annotations: meta.helm.sh/release-name: traefik meta.helm.sh/release-namespace: traefik API Version: gateway.networking.k8s.io/v1 Kind: GatewayClass Metadata: Creation Timestamp: 2024-08-14T13:42:31Z Generation: 1 Resource Version: 2120 UID: 3f0269e6-f9fa-4870-b1da-23ff17baf08d Spec: Controller Name: traefik.io/gateway-controller Status: Conditions: Last Transition Time: 2024-08-14T13:42:49Z Message: Handled by Traefik controller Observed Generation: 1 Reason: Handled Status: True Type: Accepted Events: $ kubectl describe Gateway traefik --namespace traefik Name: traefik-gateway Namespace: traefik Labels: app.kubernetes.io/instance=traefik-traefik app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=traefik helm.sh/chart=traefik-30.0.2 Annotations: meta.helm.sh/release-name: traefik meta.helm.sh/release-namespace: traefik API Version: gateway.networking.k8s.io/v1 Kind: Gateway Metadata: Creation Timestamp: 2024-08-14T13:49:05Z Generation: 1 Resource Version: 2119 UID: f287a5ae-21fe-4776-864d-142a2030895d Spec: Gateway Class Name: traefik Listeners: Allowed Routes: Namespaces: From: All Name: web Port: 8000 Protocol: HTTP Allowed Routes: Namespaces: From: All Name: websecure Port: 8443 Protocol: HTTPS Status: Conditions: Last Transition Time: 2024-08-14T13:49:05Z Message: All Listeners must be valid Observed Generation: 1 Reason: ListenersNotValid Status: False Type: Accepted Listeners: Attached Routes: 0 Conditions: Last Transition Time: 2024-08-14T13:49:05Z Message: No error found Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2024-08-14T13:49:05Z Message: No error found Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Last Transition Time: 2024-08-14T13:49:05Z Message: No error found Observed Generation: 1 Reason: Programmed Status: True Type: Programmed Name: web Supported Kinds: Group: gateway.networking.k8s.io Kind: HTTPRoute Attached Routes: 0 Conditions: Last Transition Time: 2024-08-14T13:49:05Z Message: No TLS configuration for Gateway Listener websecure:8443 and protocol "HTTPS" Observed Generation: 1 Reason: InvalidTLSConfiguration Status: False Type: Accepted Name: websecure Supported Kinds: Group: gateway.networking.k8s.io Kind: HTTPRoute Events: ``` It looks like it is listening on ports 8000 (HTTP), 8443 (HTTPS), 9000 (Traefik) and 9100 (Metrics). I'm a bit concerned about `InvalidTLSConfiguration`, `ListenersNotValid` and `No TLS configuration for Gateway Listener websecure:8443 and protocol "HTTPS"` but I think as long as we connect on port 8000 the routing should still work? My test cluster now looks like this: ```auto $ kubectl get deployments --all-namespaces NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system coredns 1/1 1 1 51m kube-system local-path-provisioner 1/1 1 1 51m kube-system metrics-server 1/1 1 1 51m traefik traefik 1/1 1 1 12m ``` Now I'm going to try and create a namespace for the whoami deployment: ```auto $ kubectl create namespace whoami namespace/whoami created ``` I create a file called `whoami.yaml` with a Deployment, Service and HTTPRoute ```auto $ cat whoami.yaml # Application to expose kind: Deployment apiVersion: apps/v1 metadata: name: whoami namespace: whoami spec: replicas: 3 selector: matchLabels: app: whoami template: metadata: labels: app: whoami spec: containers: - name: whoami image: traefik/whoami --- # Service to reach the application on the cluster apiVersion: v1 kind: Service metadata: name: whoami namespace: whoami labels: app: whoami spec: type: ClusterIP ports: - port: 80 name: whoami selector: app: whoami --- # HTTPRoute apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: whoami-httproute namespace: whoami spec: parentRefs: - name: traefik-gateway namespace: traefik hostnames: - whoami.myexample.io rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: whoami namespace: whoami port: 80 ``` And deploy it: ```auto $ kubectl apply -f whoami.yaml deployment.apps/whoami created service/whoami created httproute.gateway.networking.k8s.io/whoami-httproute created ``` My system now looks like this: ```auto $ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-576bfc4dc7-frvvd 1/1 Running 0 113m kube-system local-path-provisioner-86f46b7bf7-rtjcx 1/1 Running 0 113m kube-system metrics-server-557ff575fb-8sztf 1/1 Running 0 113m kube-system svclb-traefik-769c0052-sf9gq 2/2 Running 0 73m kube-system svclb-traefik-769c0052-wdnkg 2/2 Running 0 73m kube-system svclb-traefik-769c0052-zhf7p 2/2 Running 0 73m traefik traefik-7c7587b647-ms624 1/1 Running 0 73m whoami whoami-98d7579fb-264vv 1/1 Running 0 86s whoami whoami-98d7579fb-4txnv 1/1 Running 0 86s whoami whoami-98d7579fb-59jx2 1/1 Running 0 86s $ kubectl get deployments -A NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system coredns 1/1 1 1 113m kube-system local-path-provisioner 1/1 1 1 113m kube-system metrics-server 1/1 1 1 113m traefik traefik 1/1 1 1 73m whoami whoami 3/3 3 3 94s $ kubectl get httproute --all-namespaces NAMESPACE NAME HOSTNAMES AGE whoami whoami-httproute ["whoami.myexample.io"] 116s $ kubectl describe httproute whoami-httproute -n whoami Name: whoami-httproute Namespace: whoami Labels: Annotations: API Version: gateway.networking.k8s.io/v1 Kind: HTTPRoute Metadata: Creation Timestamp: 2024-08-14T14:54:42Z Generation: 1 Resource Version: 4273 UID: 021689ab-7e07-482d-8f43-8d854bea0009 Spec: Hostnames: whoami.myexample.io Parent Refs: Group: gateway.networking.k8s.io Kind: Gateway Name: traefik-gateway Namespace: traefik Rules: Backend Refs: Group: Kind: Service Name: whoami Namespace: whoami Port: 80 Weight: 1 Matches: Path: Type: PathPrefix Value: / Status: Parents: Conditions: Last Transition Time: 2024-08-14T14:54:48Z Message: Observed Generation: 1 Reason: Accepted Status: True Type: Accepted Last Transition Time: 2024-08-14T14:54:48Z Message: Observed Generation: 1 Reason: ResolvedRefs Status: True Type: ResolvedRefs Controller Name: traefik.io/gateway-controller Parent Ref: Group: gateway.networking.k8s.io Kind: Gateway Name: traefik-gateway Namespace: traefik Events: ``` This should complete the setup, but I don't yet understand how to access the service through the Traefik gateway. Based on these docs, it should be possible to use curl like this: ```auto $ curl -H Host:whoami.myexample.io http://10.0.0.192:8000 curl: (7) Failed to connect to 10.0.0.192 port 8000 after 4 ms: Couldn't connect to server ``` It seems that Traefik isn't actually listening on the entrypoint. I tried editing my local `/etc/hosts` file like this: ```auto 127.0.0.1 localhost 127.0.1.1 XPS-HNCG1T3 10.0.0.192 whoami.myexample.io ``` Curl still fails: ```auto $ curl http://whoami.myexample.io:8000 curl: (7) Failed to connect to whoami.myexample.io port 8000 after 4 ms: Couldn't connect to server $ dig whoami.myexample.io ; <<>> DiG 9.18.28-0ubuntu0.24.04.1-Ubuntu <<>> whoami.myexample.io ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8838 ;; flags: qr aa rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;whoami.myexample.io. IN A ;; ANSWER SECTION: whoami.myexample.io. 0 IN A 10.0.0.192 ;; Query time: 0 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP) ;; WHEN: Wed Aug 14 17:31:48 CEST 2024 ;; MSG SIZE rcvd: 64 ``` I've also tried logging in to one of the cluster nodes to see if it works from localhost: ```auto $ ssh pi@10.0.0.192 pi@turing-one:~ $ curl -H Host:whoami.myexample.io http://localhost:8000 curl: (7) Failed to connect to localhost port 8000 after 0 ms: Couldn't connect to server pi@turing-one:~ $ curl -H Host:whoami.myexample.io http://localhost:8000/ curl: (7) Failed to connect to localhost port 8000 after 0 ms: Couldn't connect to server ``` I've been stuck on this for around a week, and would greatly appreciate any help! Is there some step I am missing? Should I remove/replace the example name? How can I get this simple test scenario working? ---
**Author:** ![zespri](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/zespri/32/310_2.png) [@zespri](https://community.traefik.io/u/zespri)\ **Post date:** [August 15, 2024, 6:14am UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/2 "2024-08-15T06:14:12Z")
An interesting read. Just not to waste your time and so that you know what to expect: I have not done the traefik 3 gateway API configuration yet. This is in my to do list and I will definitely have to do it for work, but I did not get to it yet. At the same time, I'm experienced with traefik in general and have been using it in multiple configurations for years. From what you wrote it appears to me that you are not exposing the traefik ports and thus cannot connect to it. The blog post probably assumes some sort of cloud provider where you set up the port mappings between your kubernetes pod and external world in a cloud management panel, but since you are running your local copy it's up to you to expose the ports. The blog post says > The entrypoints `web` and `websecure` will be used to expose your applications. And this is true, but it is up to you to make sure that you can connect to them. As it's not possible to tell what cloud platform you are running on it also is not possible to give generic instructions in the blog post - that's my guess what happened here. So you just need to expose your traefik ports in the usual way (whichever you prefer). Once you've done that you should be good to go. (Once again, have not done it myself with v3, so there may be wrinkles I'm unaware of). ---
**Author:** ![conradwt](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/conradwt/32/9117_2.png) [@conradwt](https://community.traefik.io/u/conradwt)\ **Post date:** [August 15, 2024, 7:57am UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/3 "2024-08-15T07:57:54Z")
@strophy One needs to locate the service from the proper namespace by using the following: ```zsh kubectl get svc --namespace traefik traefik ``` The result should look something like the following: ```zsh NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE traefik LoadBalancer 10.98.135.173 192.1.2.101 80:31087/TCP,443:30107/TCP 29s ``` Now, we have an external IP address, 192.1.2.101. Thus, we can update **/etc/hosts** by appending the following to the bottom of the file: ```plaintext 192.1.2.101 whoami.myexample.io ``` Now, you can curl this domain from the command line: ```zsh ➜ curl whoami.myexample.io 404 page not found ``` Note: This is the expected message because we haven't applied our HTTPRoute resource manifest to the K8s cluster to communicate with our underlying **whoami** service. After applying the HTTPRoute resource manifest, one should see the following message: ```zsh ➜ curl whoami.myexample.io Hostname: whoami-778d7d4bdf-mkgng IP: 127.0.0.1 IP: ::1 IP: 10.244.0.3 IP: fe80::505c:f6ff:fed6:5e53 RemoteAddr: 10.244.2.2:46346 GET / HTTP/1.1 Host: whoami.myexample.io User-Agent: curl/8.9.1 Accept: */* Accept-Encoding: gzip X-Forwarded-For: 192.1.2.3 X-Forwarded-Host: whoami.myexample.io X-Forwarded-Port: 80 X-Forwarded-Proto: http X-Forwarded-Server: traefik-697c646c8b-5wwv7 X-Real-Ip: 192.1.2.3 ``` I hope that the above helps. ---
**Author:** ![strophy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/strophy/32/9119_2.png) [@strophy](https://community.traefik.io/u/strophy)\ **Post date:** [August 16, 2024, 9:08am UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/4 "2024-08-16T09:08:17Z")
@conradwt it works, thank you so much for sticking with a noob internet stranger through so many posts as I learn this! I actually had it working the whole time (I mentioned modifying the `/etc/hosts` file in my post) but incorrectly assumed that I needed to reach the service through the Traefik gateway listener port, which is configured like this: ```auto Spec: Gateway Class Name: traefik Listeners: Allowed Routes: Namespaces: From: All Name: web Port: 8000 Protocol: HTTP Allowed Routes: Namespaces: From: All Name: websecure Port: 8443 Protocol: HTTPS ``` Which doesn't work: ```auto $ curl http://whoami.myexample.io:8000 curl: (7) Failed to connect to whoami.myexample.io port 8000 after 3 ms: Couldn't connect to server ``` When in fact it was listening on port 80 this whole time: ```auto $ curl http://whoami.myexample.io Hostname: whoami-98d7579fb-59jx2 IP: 127.0.0.1 IP: ::1 IP: 10.42.1.5 IP: fe80::bc2a:5aff:fe88:dadc RemoteAddr: 10.42.3.6:40918 GET / HTTP/1.1 Host: whoami.myexample.io User-Agent: curl/8.5.0 Accept: */* Accept-Encoding: gzip X-Forwarded-For: 10.42.0.0 X-Forwarded-Host: whoami.myexample.io X-Forwarded-Port: 80 X-Forwarded-Proto: http X-Forwarded-Server: traefik-7c7587b647-ms624 X-Real-Ip: 10.42.0.0 ``` It also works when logged in to any of the k3s hosts with `curl -H Host:whoami.myexample.io http://localhost`. Just to help develop my conceptual understanding, what is purpose of the port 8000 listener, and why does it appear to not actually be listening? Thanks again mate, huge help!!! 🍻 ---
**Author:** ![conradwt](https://sea2.discourse-cdn.com/flex020/user_avatar/community.traefik.io/conradwt/32/9117_2.png) [@conradwt](https://community.traefik.io/u/conradwt)\ **Post date:** [August 16, 2024, 10:20am UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/5 "2024-08-16T10:20:25Z")
It's great to hear that things are working now. Anyway, Traefik configures port 8000 to expose an HTTP entry point. Thus, all incoming HTTP requests are routed through port 8000 to the underlying service resource. Next, the following failed because the underlying whoami service is listening on port 80 and not 8000: ```zsh curl http://whoami.myexample.io:8000 ``` Please see your K8s Gayway, HTTPRoute, and Service manifests for the details. For production environments, you should generally avoid using port 8000 directly and instead ensure that traffic is routed through HTTPS on port 443. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/containo/original/2X/b/bd81ebdb578656e76e56ff3cc3eed021d3ba132d.png) [@system](https://community.traefik.io/u/system)\ **Post date:** [August 19, 2024, 10:21am UTC](https://community.traefik.io/t/unable-to-create-working-httproute-with-traefik-3-1-0-on-k3s/24013/6 "2024-08-19T10:21:04Z")
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.