Getting real client IP (X-FORWARDED-FOR) in k3s multi-server HA setup

Hi everyone,

I've created myself the most standard K3s cluster with 3 cloud, public servers (all in "master" mode) with Traefik. I've installed cert-manager and I'm using LetsEncrypt generated wildcard SSL cert for HTTPS. I've also created a wildcard DNS record, which has all 3 public IPs in it (my aim is a proper, resilient HA cluster). There's nothing in front of the servers/traefik, no other loadbalancer, the traffic hits one of the servers directly. Now I'm having a problem with getting the real client IP in my pods (I'm using nginx inside of one of them to debug this). I'm looking at the standard X-FORWARDED-FOR and X-REAL-IP headers. I'm getting the client IP only in case the request hits the server where the Traefik pod actually runs. If it hits any of those two other servers, I get their public IP instead of the client's. I've been through google there and back and I've tried pretty much everything, including:

externalTrafficPolicy: Local

and reconfigured Traefik with:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    hostNetwork: true

    additionalArguments:
      - "--entryPoints.web.proxyProtocol.insecure"
      - "--entryPoints.web.proxyProtocol.trustedIPs=127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,IP1,IP2,IP3"
      - "--entryPoints.web.forwardedHeaders.insecure"
      - "--entrypoints.web.forwardedHeaders.trustedIPs=127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,IP1,IP2,IP3"
      - "--entryPoints.websecure.proxyProtocol.insecure"
      - "--entryPoints.websecure.proxyProtocol.trustedIPs=127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,IP1,IP2,IP3"
      - "--entryPoints.websecure.forwardedHeaders.insecure"
      - "--entrypoints.websecure.forwardedHeaders.trustedIPs=127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12,IP1,IP2,IP3"

Truth to be told I'm at the end of my options. I do not understand why Traefik behaves this way. The only solution I could come up with (which I haven't test though) is to force Traefik to run on all 3 servers, but I feel like that's an anti-pattern and it shouldn't work that way. At this point, I'm not even sure if it's bad configuration or a bug. Any advice is much appreciated.

Thank you, Jan.