Ip_whitelist.go:70 > Rejecting IP

hello world :wink:

i'm using traefik 3.0.0 and already tried both of these middlewares and got same results

traefik | 2024-05-03T02:13:17Z DBG github.com/traefik/traefik/v3/pkg/middlewares/ipallowlist/ip_allowlist.go:57 > Setting up IPAllowLister with sourceRange: [] entryPointName=websecure middlewareName=backend_ipallowlist@docker middlewareType=IPAllowLister routerName=backend@docker
traefik | 2024-05-03T02:20:08Z DBG github.com/traefik/traefik/v3/pkg/middlewares/ipwhitelist/ip_whitelist.go:70 > Rejecting IP "" matched none of the trusted IPs middlewareName=backend_ipwhitelist@docker middlewareType=IPWhiteLister is a gateway IP of a traefik container and NOT a public IP address, yet when I try to hit my website I go over Internet to reach my destination resource... up until last update, the configuration was working without any issue, and now everything broke without even touching anything((

my configuration:

# grep 'traefik\.' docker-compose.override.yaml
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_default"
      - "traefik.http.middlewares.backend_ipwhitelist.ipwhitelist.sourcerange="
      - "traefik.http.middlewares.backend_stripprefix.stripprefix.prefixes=/backend"
      - "traefik.http.routers.backend.entrypoints=websecure"
      - "traefik.http.routers.backend.middlewares=backend_stripprefix,backend_ipwhitelist"
      - "traefik.http.routers.backend.rule=Host(`X.Y.Z`) && PathPrefix(`/backend`)"
      - "traefik.http.routers.backend.tls.certresolver=myresolver"
      - "traefik.http.services.backend.loadbalancer.server.port=8000"

please advise)

Share your full Traefik static and dynamic config, and docker-compose.yml if used.

as i mentioned traefik runs within container, it's being started by docker-compose using following configuration, including traefik config:

# cat docker-compose.yaml 
    image: traefik:latest
# cat docker-compose.override.yaml 
    container_name: traefik
      - "--api.insecure=true"
      - "--certificatesresolvers.myresolver.tailscale=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.websecure.address=:443"
      - "--log.level=DEBUG"
      - "--ping=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      test: ["CMD-SHELL","traefik healthcheck --ping"]
      interval: 15s
      timeout: 5s
      retries: 2
    image: traefik:latest
      - default
      - 80:80
      - 443:443
    restart: always
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /var/run/tailscale/tailscaled.sock:/var/run/tailscale/tailscaled.sock
    image: traefik/whoami
    container_name: "simple-service"
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami.localhost`)"
      - "traefik.http.routers.whoami.entrypoints=web"

please let me know if you need anything else from me.. maybe this is a bug?

What are you trying to do? This should enable all IPs with 100.x.x.x.

i'm trying to use above middleware to limit allowed requests on the client IP, however per my original message, I'm no longer able to access that resource (even though I was able to do that in the past) and via debug, i see that Traefik rejects my request because of source IP isn't matching what I specified, however the IP address that's being presented in logs isn't my public IP address, but rather container' gateway IP..

because it was working before and just stopped on it's own without me touching any configs, i think this may be a bug.

what are your thoughts?

With sourcerange 100.* it should reject 172.*.

To me it seems there are more changes than only the update.

yes, but the issue Traefik isn't seeing my public IP, it rejects container' gateway IP address, which will always be there as next hop network wise..

i'm not aware of any other changes..

  • 100.* that's tailscale range
  • tailscale is installed on the host same as before, perhaps got an update as well
  • traefik runs within container, that runs on host
  • i'm trying to access over tailscale network (hence over 100.*)
  • traefik refuse access and in logs shows up container' gateway IP address

any ideas where to look?)

$ ssh ABC.XYZ.ts.net who
user   pts/0        2024-05-10 21:48 (
$ curl -Iv https://ABC.XYZ.ts.net/backend/docs/
* Host ABC.XYZ.ts.net:443 was resolved.
* IPv6: (none)
* IPv4:
*   Trying
* Connected to ABC.XYZ.ts.net ( port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=ABC.XYZ.ts.net
*  start date: May  1 00:31:05 2024 GMT
*  expire date: Jul 30 00:31:04 2024 GMT
*  subjectAltName: host "ABC.XYZ.ts.net" matched cert's "ABC.XYZ.ts.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://ABC.XYZ.ts.net/backend/docs/
* [HTTP/2] [1] [:method: HEAD]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: ABC.XYZ.ts.net]
* [HTTP/2] [1] [:path: /backend/docs/]
* [HTTP/2] [1] [user-agent: curl/8.5.0]
* [HTTP/2] [1] [accept: */*]
> HEAD /backend/docs/ HTTP/2
> Host: ABC.XYZ.ts.net
> User-Agent: curl/8.5.0
> Accept: */*
< HTTP/2 403 
HTTP/2 403 
< content-length: 9
content-length: 9
< date: Sat, 11 May 2024 02:07:21 GMT
date: Sat, 11 May 2024 02:07:21 GMT

* Connection #0 to host ABC.XYZ.ts.net left intact

@bluepuma77 any ideas?)

You wrote an update broke it. Can you roll back to see if it still works with an old version?

thanks again for all your time, i really appreciate that)

so, i went back, but update came with few other packages as well.. so it's hard for me to say which package caused that.. but before upgrade Traefik worked and I was using 3.0-rc5, and now there is final/stable 3.0.. i tried rc6 and rc5, got same behavior now..

i'm following docs and your suggestions, yet still unable get it to work again(

per Ip_whitelist.go:70 > Rejecting IP - #10 by alexus

you can see im' accessing my resource using tailscale network (100.x.x.x), however Traefik isn't seeing my public IP and using private IP instead and that's what's breaking whitelist(

There are multiple IPs in play here:

  1. Device IP
  2. Tailscale IP
  3. Server IP
  4. Docker IP

one can clearly see in curl verbose output that request is over Tailscale (100.x.x.x network), next Docker proxies connection into the Traefik' container, however then it appears like Traefik isn't seeing any IP other it's own gateway IP (which always will be the case as a fundamental network part).

@bluepuma77 - Do you suspect there might be a software bug in Traefik Tailscale?

I have not used tailscale, so I don’t know.