Hey all,
I made this post on the Gitlab forum, but didn't receive any response. I was wondering if anyone would have any thoughts on this.
Gut feeling
Do you think it has something to do with the Gitlab service account defined in gitlab-admin-service-account.yaml? (see below)
Issue description
I am getting the error below when I attempt to have my alpha.test container deployed via Gitlab on my K3s cluster.
Error I’m getting:
This appears in the Job logs:
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "traefik.containo.us/v1alpha1, Resource=ingressroutes", GroupVersionKind: "traefik.containo.us/v1alpha1, Kind=IngressRoute"
Name: "alpha-whoami", Namespace: "traefik"
from server for: "deployment.yml": ingressroutes.traefik.containo.us "alpha-whoami" is forbidden: User "system:serviceaccount:alpha-test-8-production:alpha-test-8-production-service-account" cannot get resource "ingressroutes" in API group "traefik.containo.us" in the namespace "traefik"
Environment
Version info
- Gitlab Version: 13.8.3 (self-managed)
- Gitlab Runner: 13.8.0
- K3s Version: 1.19.7
Steps to reproduce the issue
- Provision brand new cluster
- Connect Gitlab to cluster
- Run deployment
What is expected?
I would like my alpha.test container to deploy in its own namespace, but accept web traffic via the Ingress Controller in the traefik namespace.
Steps I have taken already to try and fix
- Hardcode the namespace into the
defaultnamespace (Gitlab kept wanting to do its own project namespace) - Delete the service account and re-add it
- Delete the cluster and re-add it (following these docs)
Additional details
Gitlab service account
I made sure I followed these steps and created the Gitlab service account with this:
Gitlab Service Account settings:
gitlab-admin-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: gitlab
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gitlab-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: gitlab
namespace: kube-system
How I installed Traefik:
I use Ansible to deploy Traefik via Helm:
ansible-playbook.yaml
---
- name: Add Traefik's chart repository.
community.kubernetes.helm_repository:
name: traefik
repo_url: "https://helm.traefik.io/traefik"
- name: Install Traefik with Helm.
community.kubernetes.helm:
name: traefik
chart_ref: traefik/traefik
update_repo_cache: yes
release_namespace: traefik
create_namespace: yes
values: "{{ lookup('template', 'traefik-chart-values.yaml') | from_yaml }}"
It calls this values file:
traefik-chart-values.yaml
#
# Advanced K8s settings
#
rbac:
enabled: true
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
- key: "node-role.kubernetes.io/master"
operator: "Exists"
effect: "NoSchedule"
#
# Configure providers
#
providers:
kubernetesIngress:
publishedService:
enabled: true
#
# Create an IngressRoute for the dashboard
#
ingressRoute:
dashboard:
enabled: true
# Configure ports
ports:
# The name of this one can't be changed as it is used for the readiness and
# liveness probes, but you can adjust its config to your liking
traefik:
port: 9000
# Use hostPort if set.
# hostPort: 9000
#
# Use hostIP if set. If not set, Kubernetes will default to 0.0.0.0, which
# means it's listening on all your interfaces and all your IPs. You may want
# to set this value if you need traefik to listen on specific interface
# only.
# hostIP: 192.168.100.10
# Defines whether the port is exposed if service.type is LoadBalancer or
# NodePort.
#
# You SHOULD NOT expose the traefik port on production deployments.
# If you want to access it from outside of your cluster,
# use `kubectl port-forward` or create a secure ingress
expose: true
# The exposed port for this service
exposedPort: 9000
# The port protocol (TCP/UDP)
protocol: TCP
web:
port: 8000
# hostPort: 8000
expose: true
exposedPort: 80
# The port protocol (TCP/UDP)
protocol: TCP
# Use nodeport if set. This is useful if you have configured Traefik in a
# LoadBalancer
# nodePort: 32080
# Port Redirections
# Added in 2.2, you can make permanent redirects via entrypoints.
# https://docs.traefik.io/routing/entrypoints/#redirection
# redirectTo: websecure
websecure:
port: 8443
# hostPort: 8443
expose: true
exposedPort: 443
# The port protocol (TCP/UDP)
protocol: TCP
# nodePort: 32443
# Set TLS at the entrypoint
# https://doc.traefik.io/traefik/routing/entrypoints/#tls
tls:
enabled: true
# this is the name of a TLSOption definition
options: ""
certResolver: ""
domains: []
# - main: example.com
# sans:
# - foo.example.com
# - bar.example.com
#
# Things that I felt more comfortable setting via the CLI
#
additionalArguments:
- "--certificatesresolvers.letsencrypt.acme.email=myemail@email.test"
- "--certificatesresolvers.letsencrypt.acme.storage=/data/acme.json"
- "--certificatesresolvers.letsencrypt.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--api.insecure=true"
- "--accesslog=true"
- "--log.level=ERROR"
My deployment
.gitlab-ci.yml
services:
- name: docker:dind
alias: docker
stages:
- deploy
deploy to production:
stage: deploy
image:
name: bitnami/kubectl:latest
entrypoint: [""]
script:
- kubectl apply -f deployment.yml
environment:
name: production
deployment.yaml
---
apiVersion: v1
kind: Pod
metadata:
name: alpha-whoami
labels:
app: alpha-whoami
namespace: alpha-test-8-production
spec:
containers:
- name: alpha-whoami
image: containous/whoami:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: alpha-whoami
namespace: alpha-test-8-production
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: alpha-whoami
type: ClusterIP
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: alpha-whoami
namespace: traefik
spec:
entryPoints:
- web
routes:
- match: Host(`alpha.test`)
kind: Rule
services:
- name: alpha-whoami
port: 80
Thank you for your help!
If there is anything else that you would like to see, just let me know! Thanks!!!
