Problems with the whitelist in GKE

Traefik Traefik v2
,
1

This is my first time creating a topic here in the community, so I ask for patience and understanding.

I just installed Traefik from scratch (v2.3.1) in a GKE cluster via helm following the documentation (https://doc.traefik.io/traefik/getting-started/install-traefik/#use-the-helm -chart), changing the following attributes:

values.yaml

service:
  spec: {
    externalTrafficPolicy: Local
  }

additionalArguments:
  - "--log.level=DEBUG"

Everything worked as expected, Traefik's Pod is running, generated a Service of type = LoadBalancer with a public IP.

Then I created an IngressRoute with an endpoint to forward to a nginx deploy with a Hello World just for testing and added a whitelist Middleware:

ing-test.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: test
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      middlewares:
        - name: whitelist
      match: Host (`xxx.xxx`)
      services:
        - name: nginx-deployment
          port: 80

whitelist.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: whitelist
spec:
  ipWhiteList:
    sourceRange:
      - 0.0.0.0/0

In the first test, access from any source was released to test the endpoint, as expected everything worked correctly. When we change Middleware for a specific IP range, we receive Forbidden:

    sourceRange:
      - 0.0.0.0/0

    sourceRange:
      - my-ip/32

Analyzing the logs of Traefik we realized that the IP that arrives in it is not the real IP of the client, but a kind of gateway of kubernetes 10.64.0.1 for example, which generates Forbbiden in all requests and we were unable to find out how to keep the real IP . We tested with Nginx Ingress and it works correctly. Does anyone have any suggestions?

GKE version:
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.12-gke.20", GitCommit:"0ac5f81eecab42bff5ef74f18b99d8896ba7b89b", GitTreeState:"clean", BuildDate:"2020-09-09T00:48:20Z", GoVersion:"go1.12.17b4", Compiler:"gc", Platform:"linux/amd64"}

1 Like
2

did you ever fix this? I am seeing the same thing

3

Hi, sorry for the delay in replying, I just got back from vacation. :beach_umbrella:

I have bad news, after a lot of insisting with Google's paid support staff (it was about 3 months) nothing was resolved.

As this is a serious problem, our only solution was to create a new cluster and then everything works again, we even approved to upgrade to the version of Traefik 2+. But this is a costly and time-consuming migration (we have around 120+ microservices, including load balancers, ingress, configs maps) we are around 70% of the migrated services but we still haven't finished, it was the only drastic solution that we found unfortunately.

If you find a better alternative, don't forget to share it here with us.

It is worth noting that it does not seem to me to be a problem with the Traefik application, after much research it seems to be related to the rules of IP tables applied to GKE nodes.

4

I've also run into this problem.

I've also concluded that it's a problem with how GKE works. For regular K8s it looks like its solvable by setting externalTrafficPolicy to "local". But this is not possible with GKE.

5

Yes, but the funny thing is that you create a new GKE cluster in the latest stable version, it will work (at least the version of traefik v2+). Maybe it was some GKE update on the network.