Traefik v2 unable to bind port 80 in k8s

Deploying Traefik with helm is failing. I am getting the error: command traefik error: error while building entryPoint web: error preparing server: error opening listener: listen tcp :80: bind: permission denied directly in the traefik pod. It is creating the service correctly though.

Here is my values.yaml with sensitive info redacted:

# Default values for Traefik
image:
  name: traefik
  tag: 2.2.1

#
# Configure the deployment
#
deployment:
  enabled: true
  # Number of pods of the deployment
  replicas: 1
  # Additional deployment annotations (e.g. for jaeger-operator sidecar injection)
  annotations: {}
  # Additional pod annotations (e.g. for mesh injection or prometheus scraping)
  podAnnotations: {}

# Create an IngressRoute for the dashboard
ingressRoute:
  dashboard:
    enabled: true
    # Additional ingressRoute annotations (e.g. for kubernetes.io/ingress.class)
    annotations: {}
    # Additional ingressRoute labels (e.g. for filtering IngressRoute by custom labels)
    labels: {}

rollingUpdate:
  maxUnavailable: 1
  maxSurge: 1

#
# Add volumes to the traefik pod.
# This can be used to mount a cert pair or a configmap that holds a config.toml file.
# After the volume has been mounted, add the configs into traefik by using the `additionalArguments` list below, eg:
# additionalArguments:
# - "--providers.file.filename=/config/dynamic.toml"
volumes: []
# - name: public-cert
#   mountPath: "/certs"
#   type: secret
# - name: configs
#   mountPath: "/config"
#   type: configMap

globalArguments:
  - "--global.checknewversion"
  - "--global.sendanonymoususage"

#
# Configure Traefik static configuration
# Additional arguments to be passed at Traefik's binary
# All available options available on https://docs.traefik.io/reference/static-configuration/cli/
## Use curly braces to pass values: `helm install --set="additionalArguments={--providers.kubernetesingress,--logs.level=DEBUG}"`
additionalArguments:
  - "--entrypoints.web.address=:80"
  - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
  - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
  - "--entrypoints.websecure.address=:443"
  - "--providers.kubernetescrd=true"
  - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=azure"
  - "--certificatesresolvers.myresolver.acme.dnschallenge.delaybeforecheck=0"
  - "--certificatesresolvers.myresolver.acme.email=<redacted>"
  - "--certificatesresolvers.myresolver.acme.storage=/data/acme.json"
  - "--certificatesresolvers.myresolver.acme.caserver=https://acme-v02.api.letsencrypt.org/directory"
  - "--certificatesresolvers.myresolver.acme.keytype=RSA4096"
  - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
#  - "--providers.kubernetesingress"
#  - "--logs.level=DEBUG"

# Environment variables to be passed to Traefik's binary
env: 
  - name: AZURE_CLIENT_ID
    value: <redacted>
  - name: AZURE_CLIENT_SECRET
    value: <redacted>
  - name: AZURE_SUBSCRIPTION_ID
    value: <redacted>
  - name: AZURE_TENANT_ID
    value: <redacted>
  - name: AZURE_RESOURCE_GROUP
    value: <redacted>

envFrom: []
# - configMapRef:
#     name: config-map-name
# - secretRef:
#     name: secret-name

# Configure ports
ports:
  # The name of this one can't be changed as it is used for the readiness and
  # liveness probes, but you can adjust its config to your liking
  traefik:
    port: 9000
    # Use hostPort if set.
    # hostPort: 9000

    # Defines whether the port is exposed if service.type is LoadBalancer or
    # NodePort.
    #
    # You SHOULD NOT expose the traefik port on production deployments.
    # If you want to access it from outside of your cluster,
    # use `kubectl proxy` or create a secure ingress
    expose: false
    # The exposed port for this service
    exposedPort: 9000
  web:
    port: 8000
    # hostPort: 8000
    expose: true
    exposedPort: 80
    # Use nodeport if set. This is useful if you have configured Traefik in a
    # LoadBalancer
    nodePort: 32080
  websecure:
    port: 8443
    # hostPort: 8443
    expose: true
    exposedPort: 443
    nodePort: 32443

# Options for the main traefik service, where the entrypoints traffic comes
# from.
service:
  enabled: true
  type: LoadBalancer
  # Additional annotations (e.g. for cloud provider specific config)
  annotations: {}
  # Additional entries here will be added to the service spec. Cannot contains
  # type, selector or ports entries.
  spec:
    # externalTrafficPolicy: Cluster
    loadBalancerIP: "<redacted>"
    # clusterIP: "2.3.4.5"
  loadBalancerSourceRanges: []
    # - 192.168.0.1/32
    # - 172.16.0.0/16
  externalIPs: []
    # - 1.2.3.4

## Create HorizontalPodAutoscaler object.
##
autoscaling:
  enabled: false
#   minReplicas: 1
#   maxReplicas: 10
#   metrics:
#   - type: Resource
#     resource:
#       name: cpu
#       targetAverageUtilization: 60
#   - type: Resource
#     resource:
#       name: memory
#       targetAverageUtilization: 60

# Enable persistence using Persistent Volume Claims
# ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
# After the pvc has been mounted, add the configs into traefik by using the `additionalArguments` list below, eg:
# additionalArguments:
# - "--certificatesresolvers.le.acme.storage=/data/acme.json"
# It will persist TLS certificates.
persistence:
  enabled: true
  # existingClaim: traefik-acme-claim

# If hostNetwork is true, runs traefik in the host network namespace
# To prevent unschedulabel pods due to port collisions, if hostNetwork=true
# and replicas>1, a pod anti-affinity is recommended and will be set if the
# affinity is left as default.
hostNetwork: false

# Additional serviceAccount annotations (e.g. for oidc authentication)
serviceAccountAnnotations: {}

resources: {}
  # requests:
  #   cpu: "100m"
  #   memory: "50Mi"
  # limits:
  #   cpu: "300m"
  #   memory: "150Mi"
affinity: {}
# # This example pod anti-affinity forces the scheduler to put traefik pods
# # on nodes where no other traefik pods are scheduled.
# # It should be used when hostNetwork: true to prevent port conflicts
#   podAntiAffinity:
#     requiredDuringSchedulingIgnoredDuringExecution:
#     - labelSelector:
#         matchExpressions:
#         - key: app
#           operator: In
#           values:
#           - {{ template "traefik.name" . }}
#       topologyKey: failure-domain.beta.kubernetes.io/zone
nodeSelector: {}
tolerations: []

# Pods can have priority.
# Priority indicates the importance of a Pod relative to other Pods.
priorityClassName: ""

# Set the container security context
# To run the container with ports below 1024 this will need to be adjust to run as root
securityContext:
  capabilities:
    drop: [ALL]
  readOnlyRootFilesystem: true
  runAsGroup: 65532
  runAsNonRoot: true
  runAsUser: 65532

podSecurityContext:
  fsGroup: 65532

Thanks in advance for the help!

should be 8000 and 8443 respectively

Thanks! I love when the fixes are easy :slight_smile:

Hi I am trying to run a fully manual setup k8s instance on a Digital Ocean VM, and I am trying to expose it using traefik. I think this means I need to bind on ports 80 and 443 using hostNetwork = True.

I was able to do this with HAProxy, using this tutorial: Getting Started | HAProxy Ingress - helm chart values are here: charts/values.yaml at master · haproxy-ingress/charts · GitHub

The service produced looks like:

Name:                     haproxy-ingress
Namespace:                default
Labels:                   app.kubernetes.io/instance=haproxy-ingress
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=haproxy-ingress
                          app.kubernetes.io/version=v0.11
                          helm.sh/chart=haproxy-ingress-0.11.0
Annotations:              meta.helm.sh/release-name: haproxy-ingress
                          meta.helm.sh/release-namespace: default
Selector:                 app.kubernetes.io/instance=haproxy-ingress,app.kubernetes.io/name=haproxy-ingress
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.105.2.56
IPs:                      10.105.2.56
Port:                     http-80  80/TCP
TargetPort:               http/TCP
NodePort:                 http-80  30118/TCP
Endpoints:                <my-digital-ocean-ip>:80
Port:                     https-443  443/TCP
TargetPort:               https/TCP
NodePort:                 https-443  31940/TCP
Endpoints:                <my-digital-ocean-ip:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     32252
Events:                   <none>

However when I try to do this in Traefik, I get the error experienced in this thread.

I have tried various value overrides, the current one that is producing this error is

helm install \
  --set hostNetwork=true \
  --set ports.web.port=80  --set ports.websecure.port=443  \
  traefik traefik/traefik

After looking deeper, I attempted to use the pod security policy to allow this, but get the same issue:

helm install \
  --set hostNetwork=true \
  --set podSecurityPolicy.enabled=true \
  --set securityContext.runAsNonRoot=false \
  --set ports.web.port=80  --set ports.websecure.port=443  \
  traefik traefik/traefik
1 Like

Hi @franco have you understood how to do it?
I have the same problem if I use helm, while I can install using the following yaml file:

---
apiVersion: v1
kind: Namespace
metadata:
  name: traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: traefik
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: traefik
  labels:
    k8s-app: traefik-ingress-lb
spec:
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
      name: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      containers:
      - image: traefik:2.4
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        # - name: admin
        #   containerPort: 8080
        #   hostPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --providers.kubernetesingress=true
        # you need to manually set this IP to the incoming public IP
        # that your ingress resources would use. Note it only affects
        # status and kubectl UI, and doesn't really do anything
        # It could even be left out https://github.com/containous/traefik/issues/6303
        - --providers.kubernetesingress.ingressendpoint.ip=<my-server-ip>
        ## uncomment these and the ports above and below to enable
        ## the web UI on the host NIC port 8080 in **insecure** mode
        - --api.dashboard=true
        - --api.insecure=true
        - --log=true
        - --log.level=INFO
        - --accesslog=true
        - --entrypoints.web.address=:80
        - --entrypoints.websecure.address=:443
        - --certificatesresolvers.leresolver.acme.tlschallenge=true # <== Enable TLS-ALPN-01 to generate and renew ACME certs
        - --certificatesresolvers.leresolver.acme.email=<email> # <== Setting email for certs
        - --certificatesresolvers.leresolver.acme.storage=/data/acme.json # <== Defining acme file to store cert information
        
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: traefik
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    # - protocol: TCP
      # port: 8080
      # name: admin
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: traefik

the values I'm using is:

  - "--certificatesresolvers.letsencrypt.acme.email=<my-email>"
  - "--certificatesresolvers.letsencrypt.acme.storage=/data/acme.json"
  - "--certificatesresolvers.letsencrypt.acme.caserver=https://acme-v02.api.letsencrypt.org/directory"
  - "--certificatesResolvers.letsencrypt.acme.tlschallenge=true"
  - "--api.insecure=true"
  - "--accesslog=true"
  - "--log.level=INFO"
hostNetwork: true
ipaddress: <my-ip>
service:
  type: ClusterIP
ports:
  web:
    port: 80
  websecure:
    port: 443

Solved, explained in my stackoverflow question/answer:

Why should it be 8000 and 8443? Cant we set port 443 and port 80 as entrypoints?

hello @anyaddres

This is done because of security reasons.
The Traefik binary inside a container is running on a non-root user, so that's why ports greater than 1024 are being used.