Traefik setup outside Kubernetes - ingress not working

Most of my setup is already running in Docker Swarm (including traefik), but I wanted to mess around with Kubernetes so I've been trying to setup that as well.

I already have a k3s cluster working, along with Rancher without issue (traefik is just pointing to the exposed 443 port).

Tried following the Kubernetes Ingress guide without luck. I don't get any errors, only confirmation of the label selector (app=traefik) and "Creating cluster-external Provider client with endpoint https://10.0.10.2:6443/" (the endpoint is a loadbalencer for my two control-nodes).

My traefik static config under the kubernetesIngress provider only has the above endpoint, my token, and the labelSelector.

Configured in Kubernetes the ClusterRole and ClusterRoleBindings as seen in the routing guide and a ingress like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
  name: nginx-test
  namespace: default
  labels:
    app: traefik
spec:
  rules:
    - host: test.zerosrealm.xyz
      http:
        paths:
          - backend:
              service:
                name: nginx
                port:
                  name: web
            path: /
            pathType: Prefix

Running k3s version v1.19.7+k3s1 if that is helpful to anyone.

Is it not possible to use a "external" (not in the kubernetes cluster) traefik? Or is my approch somehow wrong?