Unable to diagnose why requests result in 404

Traefik Traefik v2
1

Kia ora,

I'm stuck diagnosing why my requests aren't working with Traefik, but I do want to state I expect the issue to not be Traefik itself (reasonings below) but something else I can't determine due to my basic knowledge of servers, Linux, and networking. I need assistance in figuring out what is stopping Traefik from working.

Context: I am a sole developer for a team operating inside a University. The University IT department manages networking, servers, domains, etc. I have two set of servers (VMs) for two Docker Swarms, one for staging websites (early preview content), and one for production websites.

This is what the IT department has done for me for each swarm (description from IT): "A set of subdomains/DNS records and all resolve to a single BigIP Virtual IP listener on Port 443 which is then forwarding requests, unchanged, though to a single back-end node on the Docker Swarm on port 443." The BigIP manages HTTP to HTTPS, and certificates for the domains.

I also have sudo access on the swarm nodes so can do what I need on my end.

The working staging swarm

I have successfully deployed the staging swarm (1 manager with 4 workers) even though I had similar issues then. This is the compose file used to deploy Traefik as a proxy for the staging swarm:

version: '3.8'

x-default-opts:
  &default-opts
  logging:
    options:
      max-size: '1m'
      max-file: '3'

services:
  # Custom proxy to secure docker socket for Traefik
  docker-socket:
    <<: *default-opts
    image: ghcr.io/tecnativa/docker-socket-proxy:latest
    networks:
      - traefik-docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      NETWORKS: 1
      SERVICES: 1
      SWARM: 1
      TASKS: 1
    deploy:
      placement:
        constraints:
          - node.role == manager

  # Reverse proxy for handling requests
  traefik:
    <<: *default-opts
    image: traefik:2.4.13
    networks:
      - uccser-public
      - traefik-docker
    ports:
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    environment:
      - TZ=Pacific/Auckland
    command:
      # Docker
      - "--providers.docker"
      - "--providers.docker.swarmmode=true"
      - "--providers.docker.endpoint=tcp://docker-socket:2375"
      - "--providers.docker.exposedByDefault=false"
      - "--providers.docker.network=uccser-public"
      - "--providers.docker.watch"
      # Dashboard
      - "--api"
      - "--api.dashboard"
      # Previous proxy handles HTTP to HTTPS, and provides certificates
      - "--entryPoints.web.address=:443"
      # Other
      - "--log.level=DEBUG"
      - "--accesslog=true"
      - "--global.sendAnonymousUsage=false"
    deploy:
      placement:
        constraints:
            - node.role==worker
            - node.labels.role==proxy
      # Dynamic Configuration
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.dashboard.rule=Host(`${NODE_NAME}`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
        - "traefik.http.routers.dashboard.service=api@internal"
        - "traefik.http.routers.dashboard.middlewares=auth"
        - "traefik.http.middlewares.auth.basicauth.users=traefik:$$apr1$$BYRdg5Po$$1fkisZV8.aJJ6G7xR6KPz."
        - "traefik.http.services.dummy-svc.loadbalancer.server.port=9999" # Dummy service for Swarm port detection. The port can be any valid integer value.

networks:
  # This network is used by other services
  # to connect to the proxy.
  uccser-public:
    external: true
  # This network is used for Traefik to talk to
  # the Docker socket.
  traefik-docker:
    driver: overlay

The external network mentioned above is created using the following command:

docker network create --driver overlay --opt encrypted uccser-public

Multiple staging websites are hosting on this swarm with no issue:

The broken production swarm

IT has setup another set of servers for the production swarm (3 managers with 5 workers) in the same way, and I've deployed Traefik with the following file:

version: '3.8'

x-default-opts:
  &default-opts
  logging:
    options:
      max-size: '1m'
      max-file: '3'

services:
  # Custom proxy to secure docker socket for Traefik
  docker-socket:
    <<: *default-opts
    image: ghcr.io/tecnativa/docker-socket-proxy:latest
    networks:
      - traefik-docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      NETWORKS: 1
      SERVICES: 1
      SWARM: 1
      TASKS: 1
    deploy:
      placement:
        constraints:
          - node.role == manager

  # Reverse proxy for handling requests
  traefik:
    <<: *default-opts
    image: traefik:2.4.13
    networks:
      - uccser-public
      - traefik-docker
    ports:
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    environment:
      - TZ=Pacific/Auckland
    command:
      # Docker
      - "--providers.docker"
      - "--providers.docker.swarmmode=true"
      - "--providers.docker.endpoint=tcp://docker-socket:2375"
      - "--providers.docker.exposedByDefault=false"
      - "--providers.docker.network=uccser-public"
      - "--providers.docker.watch"
      # Dashboard
      - "--api"
      - "--api.dashboard"
      # Previous proxy handles HTTP to HTTPS, and provides certificates
      - "--entryPoints.web.address=:443"
      # Other
      - "--log.level=DEBUG"
      - "--accesslog=true"
      - "--accesslog.fields.headers.defaultmode=keep"
      - "--global.sendAnonymousUsage=false"
    deploy:
      placement:
        constraints:
            - node.role==worker
            - node.labels.role==proxy
      # Dynamic Configuration
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.dashboard.rule=Host(`${NODE_NAME}`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
        - "traefik.http.routers.dashboard.service=api@internal"
        - "traefik.http.routers.dashboard.middlewares=auth"
        - "traefik.http.middlewares.auth.basicauth.users=traefik:$$apr1$$BYRdg5Po$$1fkisZV8.aJJ6G7xR6KPz."
        - "traefik.http.services.dummy-svc.loadbalancer.server.port=9999" # Dummy service for Swarm port detection. The port can be any valid integer value.

networks:
  # This network is used by other services
  # to connect to the proxy.
  uccser-public:
    external: true
  # This network is used for Traefik to talk to
  # the Docker socket.
  traefik-docker:
    driver: overlay
    driver_opts:
      encrypted: "true"

And the network is created with:

docker network create --driver overlay --opt encrypted uccser-public

The only difference in the two files is the encrypted network to the Docker socket, however I can see that Traefik receives the docker configuration in the logging.

However deploying the same services as the staging swarm results in no requests being routed to them, and a 404 displayed.

What I've tried so far

The following example references a service CSANZ (GitHub repo), which is a nginx server displaying an old website. The compose file for this service is as follows:

version: '3.8'

x-default-opts:
  &default-opts
  logging:
    options:
      max-size: '1m'
      max-file: '3'

services:
    nginx:
        image: ghcr.io/uccser/csanz:master
        deploy:
            replicas: 1
            placement:
                constraints:
                    - node.role==worker
                    - node.labels.role==apps
            restart_policy:
                condition: on-failure
            labels:
                - "traefik.enable=true"
                - "traefik.docker.network=uccser-public"
                - "traefik.http.services.csanz-nginx.loadbalancer.server.port=8080"
                - "traefik.http.routers.csanz-nginx.service=csanz-nginx"
                - "traefik.http.routers.csanz-nginx.rule=Host(`www.csanz.ac.nz`)"
                - "traefik.http.routers.csanz-nginx.entryPoints=web"
        networks:
            - uccser-public

networks:
    uccser-public:
        external: true

I can use standard Docker swarm routing mesh, which involves exposing a port on the service and Docker will route a request to any node on that port to the service. So for example I've exposed our a service on port 80, so if you type any of the hostnames into a browser (on the LAN) you get the website on port 80.

When I start using the Traefik proxy, which we have on node w1 and listening on 443. So I've added a rule to the proxy to listen when the path starts with /high-school/ (will only work for one page, but it's a test). So the request to http://NODE_NAME:443/high-school/ works as expected (content only, all links are broken).

If I change the previous URL to a HTTPS request, it results in my proxy with a 404. If I try a custom domain, such as www.csanz.ac.nz it ends with a 404. The log for Traefik shows this:

proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | 132.181.106.233 - - [12/Sep/2021:06:23:26 +0000] "GET / HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" 3 "-" "-" 0ms

Here's a complete log from startup, for about 30 seconds with several 404 requests in the middle:

proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Configuration loaded from flags."
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Traefik version 2.4.13 built on 2021-07-30T15:06:29Z"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"web\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":\"10s\"},\"respondingTimeouts\":{\"idleTimeout\":\"3m0s\"}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":\"2s\",\"docker\":{\"watch\":true,\"endpoint\":\"tcp://docker-socket:2375\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmMode\":true,\"network\":\"uccser-public\",\"swarmModeRefreshSeconds\":\"15s\"}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"accessLog\":{\"format\":\"common\",\"filters\":{},\"fields\":{\"defaultMode\":\"keep\",\"headers\":{\"defaultMode\":\"keep\"}}},\"pilot\":{\"dashboard\":true}}"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Starting provider aggregator.ProviderAggregator {}"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Start TCP Server" entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Starting provider *traefik.Provider {}"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Configuration received from provider internal: {\"http\":{\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}},\"serversTransports\":{\"default\":{\"maxIdleConnsPerHost\":200}}},\"tcp\":{},\"tls\":{}}" providerName=internal
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"tcp://docker-socket:2375\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"swarmMode\":true,\"network\":\"uccser-public\",\"swarmModeRefreshSeconds\":\"15s\"}"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=info msg="Starting provider *acme.ChallengeTLSALPN {\"Timeout\":4000000000}"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Provider connection established with docker 20.10.8 (API 1.41)" providerName=docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="http: panic serving 132.181.106.231:49297: runtime error: invalid memory address or nil pointer dereference"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="goroutine 43 [running]:"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="net/http.(*conn).serve.func1(0xc000146000)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:1804 +0x153"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="panic(0x2e8af20, 0x5481b30)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/runtime/panic.go:971 +0x499"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).readClientHello(0xc000469500, 0x7efeea50f5b8, 0x18, 0xc000475ba8)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/handshake_server.go:140 +0x92"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).serverHandshake(0xc000469500, 0xc00063ce10, 0x15)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/handshake_server.go:41 +0x45"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).Handshake(0xc000469500, 0x0, 0x0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/conn.go:1391 +0xc9"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="net/http.(*conn).serve(0xc000146000, 0x399afa0, 0xc0005e81b0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:1820 +0x1a5"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="created by net/http.(*Server).Serve"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:2993 +0x39b"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Filtering disabled container" providerName=docker container=cs-field-guide-elasticsearch-kfbm9aj96a9siv8y4xt2df2b7
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Filtering disabled container" container=cs-field-guide-postgres-jymkw21r91t1xb7hqetfdn55n providerName=docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Filtering disabled container" providerName=docker container=proxy-docker-socket-v2n02mg5g7z7w88h0kqreuuoj
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"cs-field-guide-django\":{\"entryPoints\":[\"web\"],\"service\":\"cs-field-guide-django\",\"rule\":\"PathPrefix(`/chapters`)\"},\"csanz-nginx\":{\"entryPoints\":[\"web\"],\"service\":\"csanz-nginx\",\"rule\":\"Host(`NODE_NAME_w1.canterbury.ac.nz`) \\u0026\\u0026 PathPrefix(`/high-school`)\"}},\"services\":{\"cs-field-guide-django\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.43:8000\"},{\"url\":\"http://10.0.1.42:8000\"}],\"passHostHeader\":true}},\"csanz-nginx\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.30:8080\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="http: panic serving 132.181.106.232:17517: runtime error: invalid memory address or nil pointer dereference"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="goroutine 65 [running]:"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="net/http.(*conn).serve.func1(0xc0003015e0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:1804 +0x153"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="panic(0x2e8af20, 0x5481b30)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/runtime/panic.go:971 +0x499"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).readClientHello(0xc00068d500, 0x7efeea50f5b8, 0x18, 0xc000aecdb0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/handshake_server.go:140 +0x92"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).serverHandshake(0xc00068d500, 0xc00051f3e0, 0x15)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/handshake_server.go:41 +0x45"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="crypto/tls.(*Conn).Handshake(0xc00068d500, 0x0, 0x0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/crypto/tls/conn.go:1391 +0xc9"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="net/http.(*conn).serve(0xc0003015e0, 0x399afa0, 0xc0005e81b0)"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:1820 +0x1a5"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="created by net/http.(*Server).Serve"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:19+12:00" level=debug msg="\t/usr/local/golang/1.10.8/go/src/net/http/server.go:2993 +0x39b"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating middleware" middlewareType=Pipelining entryPointName=web routerName=cs-field-guide-django@docker serviceName=cs-field-guide-django middlewareName=pipelining
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating load-balancer" routerName=cs-field-guide-django@docker serviceName=cs-field-guide-django entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating server 0 http://10.0.1.43:8000" routerName=cs-field-guide-django@docker serverName=0 serviceName=cs-field-guide-django entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating server 1 http://10.0.1.42:8000" entryPointName=web routerName=cs-field-guide-django@docker serviceName=cs-field-guide-django serverName=1
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Added outgoing tracing middleware cs-field-guide-django" routerName=cs-field-guide-django@docker middlewareName=tracing middlewareType=TracingForwarder entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating middleware" middlewareType=Pipelining entryPointName=web routerName=csanz-nginx@docker serviceName=csanz-nginx middlewareName=pipelining
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating load-balancer" entryPointName=web routerName=csanz-nginx@docker serviceName=csanz-nginx
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating server 0 http://10.0.1.30:8080" routerName=csanz-nginx@docker serviceName=csanz-nginx serverName=0 entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Added outgoing tracing middleware csanz-nginx" entryPointName=web routerName=csanz-nginx@docker middlewareType=TracingForwarder middlewareName=tracing
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=web middlewareName=traefik-internal-recovery
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:20+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:23+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:24+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:25+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | 132.181.106.233 - - [12/Sep/2021:06:23:25 +0000] "GET / HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" 1 "-" "-" 0ms
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | 132.181.106.233 - - [12/Sep/2021:06:23:25 +0000] "GET / HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" 2 "-" "-" 0ms
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | 132.181.106.233 - - [12/Sep/2021:06:23:26 +0000] "GET / HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" 3 "-" "-" 0ms
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | 132.181.106.233 - - [12/Sep/2021:06:23:27 +0000] "GET / HTTP/1.1" - - "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36" 4 "-" "-" 0ms
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:28+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:29+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:33+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Filtering disabled container" providerName=docker container=cs-field-guide-elasticsearch-kfbm9aj96a9siv8y4xt2df2b7
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Filtering disabled container" container=cs-field-guide-postgres-jymkw21r91t1xb7hqetfdn55n providerName=docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Filtering disabled container" providerName=docker container=proxy-docker-socket-v2n02mg5g7z7w88h0kqreuuoj
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"cs-field-guide-django\":{\"entryPoints\":[\"web\"],\"service\":\"cs-field-guide-django\",\"rule\":\"PathPrefix(`/chapters`)\"},\"csanz-nginx\":{\"entryPoints\":[\"web\"],\"service\":\"csanz-nginx\",\"rule\":\"Host(`NODE_NAME_w1.canterbury.ac.nz`) \\u0026\\u0026 PathPrefix(`/high-school`)\"},\"dashboard\":{\"middlewares\":[\"auth\"],\"service\":\"api@internal\",\"rule\":\"Host(`NODE_NAME_w1.canterbury.ac.nz`) \\u0026\\u0026 (PathPrefix(`/api`) || PathPrefix(`/dashboard`))\"}},\"services\":{\"cs-field-guide-django\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.43:8000\"},{\"url\":\"http://10.0.1.42:8000\"}],\"passHostHeader\":true}},\"csanz-nginx\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.30:8080\"}],\"passHostHeader\":true}},\"dummy-svc\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.47:9999\"}],\"passHostHeader\":true}}},\"middlewares\":{\"auth\":{\"basicAuth\":{\"users\":[\"traefik:$apr1$BYRdg5Po$1fkisZV8.aJJ6G7xR6KPz.\"]}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="No entryPoint defined for this router, using the default one(s) instead: [web]" routerName=dashboard
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating middleware" serviceName=cs-field-guide-django middlewareName=pipelining middlewareType=Pipelining entryPointName=web routerName=cs-field-guide-django@docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating load-balancer" entryPointName=web routerName=cs-field-guide-django@docker serviceName=cs-field-guide-django
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating server 0 http://10.0.1.43:8000" serverName=0 entryPointName=web routerName=cs-field-guide-django@docker serviceName=cs-field-guide-django
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating server 1 http://10.0.1.42:8000" serviceName=cs-field-guide-django serverName=1 entryPointName=web routerName=cs-field-guide-django@docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Added outgoing tracing middleware cs-field-guide-django" entryPointName=web routerName=cs-field-guide-django@docker middlewareName=tracing middlewareType=TracingForwarder
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating middleware" middlewareType=Pipelining serviceName=csanz-nginx entryPointName=web routerName=csanz-nginx@docker middlewareName=pipelining
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating load-balancer" entryPointName=web routerName=csanz-nginx@docker serviceName=csanz-nginx
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating server 0 http://10.0.1.30:8080" serverName=0 entryPointName=web routerName=csanz-nginx@docker serviceName=csanz-nginx
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Added outgoing tracing middleware csanz-nginx" routerName=csanz-nginx@docker middlewareName=tracing middlewareType=TracingForwarder entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Added outgoing tracing middleware api@internal" middlewareName=tracing middlewareType=TracingForwarder entryPointName=web routerName=dashboard@docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating middleware" entryPointName=web middlewareName=auth@docker middlewareType=BasicAuth routerName=dashboard@docker
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Adding tracing to middleware" routerName=dashboard@docker middlewareName=auth@docker entryPointName=web
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="Creating middleware" entryPointName=web middlewareName=traefik-internal-recovery middlewareType=Recovery
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:34+12:00" level=debug msg="No default certificate, generating one"
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:38+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:39+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:43+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:44+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:48+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:49+12:00" level=debug msg="Serving default certificate for request: \"\""
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:49+12:00" level=debug msg="Filtering disabled container" providerName=docker container=cs-field-guide-elasticsearch-kfbm9aj96a9siv8y4xt2df2b7
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:49+12:00" level=debug msg="Filtering disabled container" providerName=docker container=cs-field-guide-postgres-jymkw21r91t1xb7hqetfdn55n
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:49+12:00" level=debug msg="Filtering disabled container" providerName=docker container=proxy-docker-socket-v2n02mg5g7z7w88h0kqreuuoj
proxy_traefik.1.kk6rvkwfsi3y@NODE_NAME_w1    | time="2021-09-12T18:23:49+12:00" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"cs-field-guide-django\":{\"entryPoints\":[\"web\"],\"service\":\"cs-field-guide-django\",\"rule\":\"PathPrefix(`/chapters`)\"},\"csanz-nginx\":{\"entryPoints\":[\"web\"],\"service\":\"csanz-nginx\",\"rule\":\"Host(`NODE_NAME_w1.canterbury.ac.nz`) \\u0026\\u0026 PathPrefix(`/high-school`)\"},\"dashboard\":{\"middlewares\":[\"auth\"],\"service\":\"api@internal\",\"rule\":\"Host(`NODE_NAME_w1.canterbury.ac.nz`) \\u0026\\u0026 (PathPrefix(`/api`) || PathPrefix(`/dashboard`))\"}},\"services\":{\"cs-field-guide-django\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.43:8000\"},{\"url\":\"http://10.0.1.42:8000\"}],\"passHostHeader\":true}},\"csanz-nginx\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.30:8080\"}],\"passHostHeader\":true}},\"dummy-svc\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://10.0.1.47:9999\"}],\"passHostHeader\":true}}},\"middlewares\":{\"auth\":{\"basicAuth\":{\"users\":[\"traefik:$apr1$BYRdg5Po$1fkisZV8.aJJ6G7xR6KPz.\"]}}}},\"tcp\":{},\"udp\":{}}" providerName=docker

Summary

What I believe to be working:

  • Requests to Traefik with same node hostname on any port
  • Dashboard, everything looks fine there

What I believe is not working:

  • All HTTPS requests
  • Any custom domains

What is my next step to solving this issue?

Thanks,

Jack

2

A follow up question: Is it possible to log the entire request? The current logs are showing only the path, and it would helpful to see the full request details.

I've tried modifying the logging settings, with no success.

3

Kia Ora @JackMorganNZ

First thing I am going to ask about is the F5 and what is being sent to traefik.

If the F5 is managing certificates then I believe it would be terminating TLS and sending Unencrypted requests to Traefik. I think this has to be the case as none of your configuration enables TLS. Why would you be running on 443 though ?

A little more knowledge on what is being received by Traefik will help too. Try enabling json format for the access log. This will give you a lot more info on the request including the router, scheme, this can also be done by customising the CLF, but I find the json format give most of what you want/need.

--accesslog.format=json

4

Kia ora @cakiwi,

I'm going to meet with someone from IT in a few hours to educate me about how their side works.

Switching to JSON provides way more information, didn't realise the format would have made such a difference. This is the data provided to the production swarm proxy when requesting https://cs-field-guide-prod.csse.canterbury.ac.nz/en/.

{
    "ClientAddr": "132.181.106.233:3570",
    "ClientHost": "132.181.106.233",
    "ClientPort": "3570",
    "ClientUsername": "-",
    "DownstreamContentSize": 19,
    "DownstreamStatus": 404,
    "Duration": 168326,
    "OriginContentSize": 19,
    "OriginDuration": 47824,
    "OriginStatus": 404,
    "Overhead": 120502,
    "RequestAddr": "cs-field-guide-prod.csse.canterbury.ac.nz",
    "RequestContentSize": 0,
    "RequestCount": 1,
    "RequestHost": "cs-field-guide-prod.csse.canterbury.ac.nz",
    "RequestMethod": "GET",
    "RequestPath": "/en/",
    "RequestPort": "-",
    "RequestProtocol": "HTTP/1.1",
    "RequestScheme": "http",
    "RetryAttempts": 0,
    "StartLocal": "2021-09-13T10:52:18.265816508+12:00",
    "StartUTC": "2021-09-12T22:52:18.265816508Z",
    "downstream_Content-Type": "text/plain; charset=utf-8",
    "downstream_X-Content-Type-Options": "nosniff",
    "entryPointName": "web",
    "level": "info",
    "msg": "",
    "origin_Content-Type": "text/plain; charset=utf-8",
    "origin_X-Content-Type-Options": "nosniff",
    "request_Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "request_Accept-Encoding": "gzip, deflate, br",
    "request_Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
    "request_Connection": "keep-alive",
    "request_Cookie": "PS_DEVICEFEATURES=maf:0 width:1920 height:1080 clientWidth:1920 clientHeight:945 pixelratio:1 touch:0 geolocation:1 websockets:1 webworkers:1 datepicker:1 dtpicker:1 timepicker:1 dnd:1 sessionstorage:1 localstorage:1 history:1 canvas:1 svg:1 postmessage:1 hc:0",
    "request_Dnt": "1",
    "request_Sec-Ch-Ua": "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\"",
    "request_Sec-Ch-Ua-Mobile": "?0",
    "request_Sec-Fetch-Dest": "document",
    "request_Sec-Fetch-Mode": "navigate",
    "request_Sec-Fetch-Site": "none",
    "request_Sec-Fetch-User": "?1",
    "request_Upgrade-Insecure-Requests": "1",
    "request_User-Agent": "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36",
    "request_X-Forwarded-Host": "cs-field-guide-prod.csse.canterbury.ac.nz",
    "request_X-Forwarded-Port": "80",
    "request_X-Forwarded-Proto": "http",
    "request_X-Forwarded-Server": "5cc96a36b81f",
    "request_X-Real-Ip": "132.181.106.233",
    "time": "2021-09-13T10:52:18+12:00"
}

Regarding TLS and port 443: The port number was picked by IT, but could be anything. I couldn't find any documentation that stated Traefik handles port 443 any differently to any other port, but I agree using that one is confusing.

5

What I see is missing is that there is no RouterName or ServiceName present in the accesslog line. This would indeed indicate there is no route being matched.

I didn't look at your log earlier, I'll have a look now.

6

Yep, I compared it to a similar request on our staging website that works, you can see the service working:

{
    "ClientAddr": "132.181.106.233:15934",
    "ClientHost": "132.181.106.233",
    "ClientPort": "15934",
    "ClientUsername": "-",
    "DownstreamContentSize": 41104,
    "DownstreamStatus": 200,
    "Duration": 55144296,
    "OriginContentSize": 41104,
    "OriginDuration": 55065997,
    "OriginStatus": 200,
    "Overhead": 78299,
    "RequestAddr": "cs-field-guide-dev.csse.canterbury.ac.nz",
    "RequestContentSize": 0,
    "RequestCount": 10,
    "RequestHost": "cs-field-guide-dev.csse.canterbury.ac.nz",
    "RequestMethod": "GET",
    "RequestPath": "/en/",
    "RequestPort": "-",
    "RequestProtocol": "HTTP/1.1",
    "RequestScheme": "http",
    "RetryAttempts": 0,
    "RouterName": "cs-field-guide-django@docker",
    "ServiceAddr": "10.0.35.21:8000",
    "ServiceName": "cs-field-guide-django@docker",
    "ServiceURL": {
        "Scheme": "http",
        "Opaque": "",
        "User": null,
        "Host": "10.0.35.21:8000",
        "Path": "",
        "RawPath": "",
        "ForceQuery": false,
        "RawQuery": "",
        "Fragment": "",
        "RawFragment": ""
    },
    "StartLocal": "2021-09-13T11:10:19.90673013+12:00",
    "StartUTC": "2021-09-12T23:10:19.90673013Z",
    "downstream_Content-Language": "en",
    "downstream_Content-Length": "41104",
    "downstream_Content-Type": "text/html; charset=utf-8",
    "downstream_Date": "Sun, 12 Sep 2021 23:10:19 GMT",
    "downstream_Referrer-Policy": "same-origin",
    "downstream_Server": "gunicorn",
    "downstream_Vary": "Cookie",
    "downstream_X-Content-Type-Options": "nosniff",
    "downstream_X-Frame-Options": "DENY",
    "entryPointName": "web",
    "level": "info",
    "msg": "",
    "origin_Content-Language": "en",
    "origin_Content-Length": "41104",
    "origin_Content-Type": "text/html; charset=utf-8",
    "origin_Date": "Sun, 12 Sep 2021 23:10:19 GMT",
    "origin_Referrer-Policy": "same-origin",
    "origin_Server": "gunicorn",
    "origin_Vary": "Cookie",
    "origin_X-Content-Type-Options": "nosniff",
    "origin_X-Frame-Options": "DENY",
    "request_Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "request_Accept-Encoding": "gzip, deflate, br",
    "request_Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
    "request_Cache-Control": "max-age=0",
    "request_Connection": "keep-alive",
    "request_Cookie": "PS_DEVICEFEATURES=maf:0 width:1920 height:1080 clientWidth:1920 clientHeight:881 pixelratio:1 touch:0 geolocation:1 websockets:1 webworkers:1 datepicker:1 dtpicker:1 timepicker:1 dnd:1 sessionstorage:1 localstorage:1 history:1 canvas:1 svg:1 postmessage:1 hc:0; ",
    "request_Dnt": "1",
    "request_Sec-Ch-Ua": "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\"",
    "request_Sec-Ch-Ua-Mobile": "?0",
    "request_Sec-Fetch-Dest": "document",
    "request_Sec-Fetch-Mode": "navigate",
    "request_Sec-Fetch-Site": "none",
    "request_Sec-Fetch-User": "?1",
    "request_Upgrade-Insecure-Requests": "1",
    "request_User-Agent": "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36",
    "request_X-Forwarded-Host": "cs-field-guide-dev.csse.canterbury.ac.nz",
    "request_X-Forwarded-Port": "80",
    "request_X-Forwarded-Proto": "http",
    "request_X-Forwarded-Server": "3f3b08f39131",
    "request_X-Real-Ip": "132.181.106.233",
    "time": "2021-09-13T11:10:19+12:00"
}
7

Looking at the config loaded by the docker provider you get:

docker_config 

  "http": {
    "routers": {
      "cs-field-guide-django": {
        "entryPoints": [
          "web"
        ],
        "service": "cs-field-guide-django",
        "rule": "PathPrefix(`/chapters`)"
      },
      "csanz-nginx": {
        "entryPoints": [
          "web"
        ],
        "service": "csanz-nginx",
        "rule": "Host(`NODE_NAME_w1.canterbury.ac.nz`) && PathPrefix(`/high-school`)"
      },
      "dashboard": {
        "middlewares": [
          "auth"
        ],
        "service": "api@internal",
        "rule": "Host(`NODE_NAME_w1.canterbury.ac.nz`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))"
      }
    },
    "services": {
      "cs-field-guide-django": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://10.0.1.43:8000"
            },
            {
              "url": "http://10.0.1.42:8000"
            }
          ],
          "passHostHeader": true
        }
      },
      "csanz-nginx": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://10.0.1.30:8080"
            }
          ],
          "passHostHeader": true
        }
      },
      "dummy-svc": {
        "loadBalancer": {
          "servers": [
            {
              "url": "http://10.0.1.47:9999"
            }
          ],
          "passHostHeader": true
        }
      }
    },
    "middlewares": {
      "auth": {
        "basicAuth": {
          "users": [
            "traefik:$apr1$BYRdg5Po$1fkisZV8.aJJ6G7xR6KPz."
          ]
        }
      }
    }
  },
  "tcp": {},
  "udp": {}
}

So based on the rules listed there would not be a match for that request.

1 Like
8

Thanks for pointing that out, I had a test configuration still loaded for that service that I must have forgotten about, so I've reverted that. The logs now show the same request being routed properly.

I am still having issues with the other domain, but turns out all the certificates haven't been finished yet on the IT side, hence why it's not resolving at the moment. I'll update this post once these are finished.

9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.