Service blocked by intranet firewall

Traefik Traefik v3 (latest)
1

I am hosting a web wikijs service behind Traefik on the intranet to get a friendly hostname. both wikijs and Traefik are inside a docker container

let's say the host server ip is 192.268.3.10. wikijs service is on 4000.

When I am access the service through host name it's blocked by the corporate firewall, however, it's not through ip.

Apparently, Traefik send my request not directly to the host ip(192.168.3.10), but through the firewall.

this only happened lately when I am use v3.0, How did this happen? how can I fix it?

tests

$  wget http://wiki.office/login
--2024-11-28 12:53:22--  http://wiki.office/login
Resolving wiki.office (wiki.office)... 127.0.1.1
Connecting to wiki.office (wiki.office)|127.0.1.1|:80... connected.
HTTP request sent, awaiting response... 407 Proxy Authentication Required
2024-11-28 12:53:23 ERROR 407: Proxy Authentication Required.

$ wget http://192.268.3.10:4000/login
--2024-11-28 12:54:03--  http://192.268.3.10:4000/login
Connecting to 192.268.3.10:4000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2174 (2.1K) [text/html]
Saving to: ‘login’

login                                         100%[=====================================================>]   2.12K  --.-KB/s    in 0s

2024-11-28 12:54:03 (385 MB/s) - ‘login’ saved [2174/2174]

docker compose files

  • traefik
services:
  traefik:
    image: "traefik:v3.0"
    container_name: "traefik"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--entrypoints.web.address=:80"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.file.watch=true"
      - "--providers.file.directory=/dynamic-config"
      - "--metrics.prometheus=true"
      - "--entryPoints.metrics.address=:8082"
      - "--metrics.prometheus.entryPoint=metrics"

      - "--accesslog=true"
      - "--accesslog.filepath=/etc/traefik/access.log"
      - "--accesslog.format=json"
      - "--accesslog.bufferingsize=100"
      - "--accesslog.filters.statuscodes=200,300-302"
      - "--accesslog.filters.retryattempts"
      - "--accesslog.filters.minduration=10ms"
      - "--accesslog.fields.defaultmode=keep"
      - "--accesslog.fields.names.ClientUsername=drop"
      - "--accesslog.fields.headers.defaultmode=keep"
      - "--accesslog.fields.headers.names.User-Agent=redact"
      - "--accesslog.fields.headers.names.Authorization=drop"
      - "--accesslog.fields.headers.names.Content-Type=keep"

      # $ curl localhost:8080/metrics
      - "--metrics.prometheus=true"
      - "--metrics.prometheus.buckets=0.100000, 0.300000, 1.200000, 5.000000"
      - "--metrics.prometheus.addEntryPointsLabels=true"
      - "--metrics.prometheus.addServicesLabels=true"
      - "--metrics.prometheus.addrouterslabels=true"


    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "~/docker-mapping/traefik:/etc/traefik"
      - "./traefik-dyna-config:/dynamic-config"
    ports:
      - "80:80"
      - "8800:8080"
      - "8882:8082"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.office`) || Host(`hub.office`) "
      - "traefik.http.routers.traefik.entrypoints=web"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"
  • wikijs
services:
  wikijs:
    image: lscr.io/linuxserver/wikijs:2.5.305
    container_name: wikijs
    ports:
      - 4000:3000
    restart: always
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.wikijs.rule=Host(`wiki.office`) "
      - "traefik.http.routers.wikijs.entrypoints=web"
      - "traefik.http.services.wikijs.loadbalancer.server.port=3000"
2

is not something Traefik by itself provides. The domain name is provided by DNS service (local, corporate, global) or from hosts file. Your browser will resolve the domain name to IP first, then connect to the IP (and port).

3

Actually it does have something to do with Traefik. Here's my tests:


$ wget wiki.office
--2024-11-29 08:30:56--  http://wiki.office/
Resolving wiki.office (wiki.office)... 192.268.3.10
Connecting to wiki.office (wiki.office)|192.268.3.10|:80... connected.
HTTP request sent, awaiting response... 407 Proxy Authentication Required
2024-11-29 08:30:57 ERROR 407: Proxy Authentication Required.

$ wget wiki.office:4000
--2024-11-29 08:31:05--  http://wiki.office:4000/
Resolving wiki.office (wiki.office)... 192.268.3.10
Connecting to wiki.office (wiki.office)|192.268.3.10|:4000... connected.
HTTP request sent, awaiting response... 302 Found
Location: /en/home [following]
--2024-11-29 08:31:05--  http://wiki.office:4000/en/home
Reusing existing connection to wiki.office:4000.
HTTP request sent, awaiting response... 302 Found
Location: /login [following]
--2024-11-29 08:31:05--  http://wiki.office:4000/login
Reusing existing connection to wiki.office:4000.
HTTP request sent, awaiting response... 200 OK
Length: 2174 (2.1K) [text/html]
Saving to: ‘index.html’

index.html  100%[==================>]   2.12K  --.-KB/s    in 0s      

2024-11-29 08:31:05 (150 MB/s) - ‘index.html’ saved [2174/2174]

$ wget 192.268.3.10:4000
--2024-11-29 08:31:58--  http://192.268.3.10:4000/
Connecting to 192.268.3.10:4000... connected.
HTTP request sent, awaiting response... 302 Found
Location: /en/home [following]
--2024-11-29 08:31:58--  http://192.268.3.10:4000/en/home
Reusing existing connection to 192.268.3.10:4000.
HTTP request sent, awaiting response... 302 Found
Location: /login [following]
--2024-11-29 08:31:58--  http://192.268.3.10:4000/login
Reusing existing connection to 192.268.3.10:4000.
HTTP request sent, awaiting response... 200 OK
Length: 2174 (2.1K) [text/html]
Saving to: ‘index.html.1’

index.html.1  100%[================>]   2.12K  --.-KB/s    in 0s      

2024-11-29 08:31:58 (643 MB/s) - ‘index.html.1’ saved [2174/2174]

$ wget 192.268.3.10
--2024-11-29 08:32:05--  http://192.268.3.10/
Connecting to 192.268.3.10:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-11-29 08:32:05 ERROR 404: Not Found.

Here's the path of the request of wget wiki.office

wget http request -> client `hosts` file -> server host(192.268.3.10) -> Traefik router(port 80) ->(internals I don't know) ->Traefik service(wiki.office:4000) -> wikijs at port 4000
  1. wget wiki.office:4000 being ok verifies the first and last part are OK.
  • wget http request -> client's hosts file -> server host(192.268.3.10), client's hosts file works as expected
  • wikijs at port 4000, wiki service is running too
  • only Traefik is bypassed
  1. wget 192.268.3.10 being ok verifies
  • port 80 on Traefik's docker host is fine too, i.e. the http request reaches Traefik's http entrypoint(port 80) and enters Traefik.

both 1 & 2 shows the traffic remain inside intranet, without triggering the firewall.

wget wiki.office trigger the firewall shows when the request enters Traefik from port 80, and somehow in the (internals I don't know) part, it routes to the firewall
- -> Traefik router(port 80) ->(internals I don't know) ->Traefik service(wiki.office:4000)

the following has been added to .bashrc the docker host(i.e. 192.268.3.10) with no help. it could be Traefik being running inside the docker.

export no_proxy="wiki.advme.global"

Can anynoe explain the mechaniasm of (internals I don't know) and point to a solution?

4

What are Traefik debug log and Traefik access log (in JSON format) telling you about the requests?

I don’t see any possibility that Traefik is using a proxy server when connecting to a target service.

It’s probably rather wget using the proxy server, configured on your local host, and that responds with the proxy auth. Try wget -d http://example.com for debug info.