Pretty much mostly just pulled this straight from the bug report rather than typing it in again.
What version of Traefik are you using?
2.9.4
What did you do?
I've installed k3s with the goal of eventually moving a lot of my currently co-lo hosted services over to it and so far, so good, the setup is "air-gap" style with traefik (as installed by k3s) being used with ingress behind a load balancer provided by hetzner, I've reached the point where I'm adding some non http/https services but I'm having an issue getting the new entry points detected by the load balancer.
What did you see instead?
The load balancer is not picking up anything on the exposed ports, traefik is showing the entrypoints in the dashboard and there are no errors in the logs.
What is your environment & configuration?
The configuration for the entrypoints are installed via /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
additionalArguments:
- "--entryPoints.web.proxyProtocol.trustedIPs=10.0.0.6,10.42.0.0/16"
- "--entryPoints.web.forwardedHeaders.trustedIPs=10.0.0.6,10.42.0.0/16"
- "--entryPoints.websecure.proxyProtocol.trustedIPs=10.0.0.6,10.42.0.0/16"
- "--entryPoints.websecure.forwardedHeaders.trustedIPs=10.0.0.6,10.42.0.0/16"
- "--entryPoints.ssh.address=:10022/tcp"
- "--entryPoints.smtp.address=:10025/tcp""
ports:
smtp:
port: 10025
exposed: true
exposedPort: 10025
protocol: TCP
ssh:
port: 10022
exposed: true
exposedPort: 10022
protocol: TCP
The test service is as follows.
apiVersion: apps/v1
kind: Deployment
metadata:
name: postfix
labels:
chart: generic-service
app: postfix
spec:
selector:
matchLabels:
app: postfix
replicas: 1
template:
metadata:
labels:
app: postfix
spec:
serviceAccountName: default
containers:
- name: postfix
image: 10.0.0.5:5001/postfix
ports:
- name: postfix-25
containerPort: 25
protocol: TCP
volumeMounts:
- name: core
mountPath: /opt/mailman
- name: config
mountPath: /etc/postfix
volumes:
- name: core
hostPath:
path: /mnt/gluster-k8s/mailman/core
- name: config
hostPath:
path: /mnt/gluster-k8s/postfix/etc
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "postfix"
Source: generic-service/templates/ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
annotations:
kubernetes.io/ingress.class: traefik
name: postfix-main
labels:
app: postfix
spec:
entryPoints:
- smtp
routes:
match: "HostSNI(*)"
services:
name: postfix-25
port: 25
Pods and services
NAMESPACE NAME READY STATUS RESTARTS AGE
default postfix-7df978896b-vnbxg 1/1 Running 0 2d9h
cert-manager cert-manager-webhook-5594f5c99b-ft86m 1/1 Running 0 40h
kube-system helm-install-traefik-crd-66jwv 0/1 Completed 0 39h
kube-system metrics-server-76d86976f8-7xxf6 1/1 Running 0 40h
kube-system svclb-traefik-d10f3dca-sddjv 2/2 Running 0 38h
kube-system svclb-traefik-d10f3dca-z2k4k 2/2 Running 0 38h
kube-system svclb-traefik-d10f3dca-s98fg 2/2 Running 0 38h
kube-system traefik-b65bd5d48-ffj9h 1/1 Running 0 38h
kube-system helm-install-traefik-86fst 0/1 Completed 0 29h
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default postfix-25 ClusterIP 10.43.164.98 <none> 25/TCP 2d9h
kube-system traefik LoadBalancer 10.43.86.185 10.0.0.2,10.0.0.3,10.0.0.4 80:31613/TCP,443:32666/TC
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 70d
kube-system kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 70d
kube-system metrics-server ClusterIP 10.43.7.209 <none> 443/TCP 70d
default webtest1 ClusterIP 10.43.218.87 <none> 8001/TCP 68d
cert-manager cert-manager-webhook ClusterIP 10.43.171.227 <none> 443/TCP 47d
cert-manager cert-manager ClusterIP 10.43.181.178 <none> 9402/TCP 47d
default php-test-443 ClusterIP 10.43.201.144 <none> 443/TCP 36d
default php-test-80 ClusterIP 10.43.43.43 <none> 80/TCP 36d
default ant-443 ClusterIP 10.43.116.113 <none> 443/TCP 35d
default ant-80 ClusterIP 10.43.39.19 <none> 80/TCP 35d
default webhook-9000 ClusterIP 10.43.56.82 <none> 9000/TCP 34d
default sshtest1 ClusterIP 10.43.68.75 <none> 10022/TCP 67d
default mailman-8024 ClusterIP 10.43.155.153 <none> 8024/TCP 18d
default mailman-80 ClusterIP 10.43.167.122 <none> 80/TCP 18d
default postfix-25 ClusterIP 10.43.253.211 <none> 25/TCP 2d14h
kube-system traefik LoadBalancer 10.43.218.88 10.0.0.2,10.0.0.3,10.0.0.4 80:32696/TCP,443:32395/TCP 4d12h
And the results of describing the deployment
Name: traefik
Namespace: kube-system
CreationTimestamp: Wed, 31 May 2023 00:09:17 +0000
Labels: app.kubernetes.io/instance=traefik-kube-system
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=traefik
helm.sh/chart=traefik-21.2.1_up21.2.0
Annotations: deployment.kubernetes.io/revision: 18
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: kube-system
Selector: app.kubernetes.io/instance=traefik-kube-system,app.kubernetes.io/name=traefik
Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 0 max unavailable, 1 max surge
Pod Template:
Labels: app.kubernetes.io/instance=traefik-kube-system
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=traefik
helm.sh/chart=traefik-21.2.1_up21.2.0
Annotations: prometheus.io/path: /metrics
prometheus.io/port: 9100
prometheus.io/scrape: true
Service Account: traefik
Containers:
traefik:
Image: rancher/mirrored-library-traefik:2.9.10
Ports: 9100/TCP, 10025/TCP, 10022/TCP, 10024/TCP, 9000/TCP, 8000/TCP, 8443/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
Args:
--global.checknewversion
--global.sendanonymoususage
--entrypoints.metrics.address=:9100/tcp
--entrypoints.smtp.address=:10025/tcp
--entrypoints.ssh.address=:10022/tcp
--entrypoints.test.address=:10024/tcp
--entrypoints.traefik.address=:9000/tcp
--entrypoints.web.address=:8000/tcp
--entrypoints.websecure.address=:8443/tcp
--api.dashboard=true
--ping=true
--metrics.prometheus=true
--metrics.prometheus.entrypoint=metrics
--providers.kubernetescrd
--providers.kubernetesingress
--providers.kubernetesingress.ingressendpoint.publishedservice=kube-system/traefik
--entrypoints.websecure.http.tls=true
--entryPoints.web.proxyProtocol.trustedIPs=10.0.0.6,10.42.0.0/16
--entryPoints.web.forwardedHeaders.trustedIPs=10.0.0.6,10.42.0.0/16
--entryPoints.websecure.proxyProtocol.trustedIPs=10.0.0.6,10.42.0.0/16
--entryPoints.websecure.forwardedHeaders.trustedIPs=10.0.0.6,10.42.0.0/16
--log.level=DEBUG
Liveness: http-get http://:9000/ping delay=2s timeout=2s period=10s #success=1 #failure=3
Readiness: http-get http://:9000/ping delay=2s timeout=2s period=10s #success=1 #failure=1
Environment: <none>
Mounts:
/data from data (rw)
/tmp from tmp (rw)
Volumes:
data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
tmp:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Priority Class Name: system-cluster-critical
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: traefik-54645fb756 (1/1 replicas created)
Events: <none>