IPAllowList doesn't work with Cloudflare proxy

I have setup like this:
Client -> VPN -> Cloudflare Proxy -> Digital Ocean Loadbalancer -> Traefik -> Container

So only the client with VPN enabled can access the resource hidden by middleware with IPAllowList under the hood with VPN IP specified.

I've specified trusted IPs list contains IP of my DO load balancer and CloudFlare proxy IPs

proxyProtocol:
  trustedIPs:
    {{- range .Values.loadBalancerTrustedIps }}
      - {{ . }}
    {{- end }}
forwardedHeaders:
  trustedIPs:
    {{- range .Values.loadBalancerTrustedIps }}
      - {{ . }}
    {{- end }}

Middleware with IPAllowList

ipAllowList:
  sourceRange:
    {{- range .Values.vpnIPs }}
      - {{ . }}
    {{- end }}

The setup works without CloudFlare proxy enabled, like:
Client -> VPN -> Digital Ocean Loadbalancer -> Traefik -> Container

but when it's enabled I am getting

Rejecting IP CloudFlare Proxy IP: CloudFlare Proxy IP matched none of the trusted IPs

So it looks like trusted ip is not working, but when I look into access logs I can see my VPN IP in X-Forwarded-For header, which as I understand from the doc should be utilised under the hood of IPAllowList. X-Forwarded-For header was not used to determine client IP, but the X-Real-Ip header was (as I can see from the log)

"ClientAddr": "<CloudFlare Proxy IP>:<Port>",
"ClientHost": "<My VPN IP>",
...
"request_X-Forwarded-For": <My VPN IP>,
...
"request_X-Forwarded-Port": "443",
"request_X-Forwarded-Proto": "https",
...
"request_X-Real-Ip": <CloudFlare Proxy IP>,
...

Why doesn't it use X-Forwarded-For header like doc describes, but uses X-Real-Ip header?