Need help understanding debug logs

Traefik Traefik v2 (latest)
,
1

I'm trying to set up a production server with Docker and Traefik. I've been running into issues so I'm currently testing with the basic Traefik + Whoami containers. When trying to set up global redirects I keep running into issues with the server suddenly processing thousands of requests and eventually getting OOM errors and a 25GB+ log file.

I have two domains on DigitalOcean, starthubs.co.uk and starthubs.uk, and I want http://*, .co.uk and www. to be redirected to https://starthubs.uk, and allowing traefik dashboard access on https://traefik.starthubs.uk.

I haven't been able to get all of the redirects working, as once I add the subdomain and .co.uk redirects I start to get thousands of log messages per second starting between a minute and an hour after running the server.

I have had all my Traefik global redirects written at some point but have stripped it down in the following code to the simplest version that causes issues to try and troubleshoot. This version just redirects from http and replaces .co.uk with .uk, keeping the subdomains. It works as expected other than the random access logs occuring.

version: "3.7"

services:
  traefik:
    image: traefik:v2.2
    restart: always
    security_opt:
      - no-new-privileges:true
    command:
      - "--api=true"
      - "--api.insecure=false"
      - "--api.debug=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - '--providers.docker.defaultRule=Host(`{{ index .Labels "com.docker.compose.service" }}.localhost`)'
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--serverstransport.maxIdleConnsPerHost=5"
      - "--serversTransport.forwardingTimeouts.idleConnTimeout=60"
      - "--certificatesresolvers.leresolver.acme.email=emailaddress@gmail.com"
      - "--certificatesresolvers.leresolver.acme.storage=/letsencrypt/acme.json"
      - "--certificatesresolvers.leresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.leresolver.acme.dnschallenge.provider=digitalocean"
      - "--log.level=DEBUG"
    ports:
      - "80:80"
      - "443:443"
    environment:
      - "DO_AUTH_TOKEN={auth token goes here}"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    labels:
      # Dashboard
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.rule=Host(`traefik.starthubs.uk`)"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.routers.traefik.tls.certresolver=leresolver"
      - "traefik.http.routers.traefik.tls=true"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.middlewares=authtraefik"
      - "traefik.http.middlewares.authtraefik.basicauth.users=user:$$apr1$$ftx46sm9$$GHtH3awnQiO.7OL29Ydwa."
     
      # redirect middleware
      - "traefik.http.middlewares.test.redirectregex.regex=https?:\\/\\/(.*\\.)?starthubs(?:\\.co)?\\.uk(.*)?"
      - "traefik.http.middlewares.test.redirectregex.replacement=https://$${1}starthubs.uk$${2}"
      - "traefik.http.middlewares.test.redirectregex.permanent=true"

      # global redirect
      - "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=test"

      # global wildcard certificates
      - 'traefik.http.routers.traefik.tls.domains[0].main=starthubs.uk'
      - 'traefik.http.routers.traefik.tls.domains[0].sans=*.starthubs.uk'
      - 'traefik.http.routers.traefik.tls.domains[1].main=starthubs.co.uk'
      - 'traefik.http.routers.traefik.tls.domains[1].sans=*.starthubs.co.uk'


  whoami:
    image: containous/whoami
    hostname: whoami
    ports:
      - 80
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.whoami.loadbalancer.server.port=80"
      - "traefik.http.routers.whoami.rule=Host(`starthubs.uk`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls=true"
      - "traefik.http.routers.whoami.tls.certresolver=leresolver"

Here's the top few lines of the log, there were thousands more lines before I interrupted the process but they all repeat the last few entries (Pastebin link with more lines). I'm not sure if this is just a DDOS attack or if it's an issue with my config. After restarting the server I will get the requests from a different IP address each time, but quite often Iran based IPs. When I simplify the redirect code to not include any subdomain redirects I do not seem to experience this issue even when running the server for many hours, so I am not convinced it is a DDOS. I have tried a lot of different config options and I'm out of ideas here, so any help is greatly appreciated.

The logs mention vshop.mdc.gameloft.com, and my last test before this one referenced ubsoundmiamihao.com in its place. I do not know either of these domains and they are not associated with my IP, but from my understanding of the log the request is trying to POST to them?

traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Configuration loaded from flags."
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Traefik version 2.2.1 built on 2020-04-29T18:02:09Z"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":5,\"forwardingTimeouts\":{\"dialTimeout\":30000000000,\"idleConnTimeout\":60000000000}},\"entryPoints\":{\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ index .Labels \\\"com.docker.compose.service\\\" }}.localhost`)\",\"swarmModeRefreshSeconds\":15000000000}},\"api\":{\"dashboard\":true,\"debug\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"leresolver\":{\"acme\":{\"email\":\"emailaddress@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"dnsChallenge\":{\"provider\":\"digitalocean\"}}}}}"
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/contributing/data-collection/\n"
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Start TCP Server" entryPointName=web
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Start TCP Server" entryPointName=websecure
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Starting provider *acme.Provider {\"email\":\"emailaddress@gmail.com\",\"caServer\":\"https://acme-v02.api.letsencrypt.org/directory\",\"storage\":\"/letsencrypt/acme.json\",\"keyType\":\"RSA4096\",\"dnsChallenge\":{\"provider\":\"digitalocean\"},\"ResolverName\":\"leresolver\",\"store\":{},\"ChallengeStore\":{}}"
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Testing certificate renew..." providerName=leresolver.acme
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ index .Labels \\\"com.docker.compose.service\\\" }}.localhost`)\",\"swarmModeRefreshSeconds\":15000000000}"
traefik_1  | time="2020-05-24T01:44:31Z" level=info msg="Starting provider *traefik.Provider {}"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Configuration received from provider leresolver.acme: {\"http\":{},\"tls\":{}}" providerName=leresolver.acme
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Configuration received from provider internal: {\"http\":{\"services\":{\"api\":{},\"dashboard\":{},\"noop\":{}}},\"tcp\":{},\"tls\":{}}" providerName=internal
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.co.uk,*.starthubs.co.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk,*.starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No default certificate, generating one"
whoami_1   | Starting up on port 80
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Provider connection established with docker 19.03.1 (API 1.40)" providerName=docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"http-catchall\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"test\"],\"service\":\"traefik-starthubs\",\"rule\":\"hostregexp(`{host:.+}`)\"},\"traefik\":{\"entryPoints\":[\"websecure\"],\"middlewares\":[\"authtraefik\"],\"service\":\"api@internal\",\"rule\":\"Host(`traefik.starthubs.uk`)\",\"tls\":{\"certResolver\":\"leresolver\",\"domains\":[{\"main\":\"starthubs.uk\",\"sans\":[\"*.starthubs.uk\"]},{\"main\":\"starthubs.co.uk\",\"sans\":[\"*.starthubs.co.uk\"]}]}},\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami\",\"rule\":\"Host(`starthubs.uk`)\",\"tls\":{\"certResolver\":\"leresolver\"}}},\"services\":{\"traefik-starthubs\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.21.0.2:80\"}],\"passHostHeader\":true}},\"whoami\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.21.0.3:80\"}],\"passHostHeader\":true}}},\"middlewares\":{\"authtraefik\":{\"basicAuth\":{\"users\":[\"user:$apr1$ftx46sm9$GHtH3awnQiO.7OL29Ydwa.\"]}},\"test\":{\"redirectRegex\":{\"regex\":\"https?:\\\\/\\\\/(.*\\\\.)?starthubs(?:\\\\.co)?\\\\.uk(.*)?\",\"replacement\":\"https://${1}starthubs.uk${2}\",\"permanent\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.co.uk,*.starthubs.co.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk,*.starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.co.uk,*.starthubs.co.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding certificate for domain(s) starthubs.uk,*.starthubs.uk"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" routerName=http-catchall@docker serviceName=traefik-starthubs middlewareName=pipelining middlewareType=Pipelining entryPointName=web
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=http-catchall@docker serviceName=traefik-starthubs
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating server 0 http://172.21.0.2:80" entryPointName=web routerName=http-catchall@docker serviceName=traefik-starthubs serverName=0
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Added outgoing tracing middleware traefik-starthubs" entryPointName=web routerName=http-catchall@docker middlewareType=TracingForwarder middlewareName=tracing
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" middlewareType=RedirectRegex entryPointName=web routerName=http-catchall@docker middlewareName=test@docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Setting up redirection from https?:\\/\\/(.*\\.)?starthubs(?:\\.co)?\\.uk(.*)? to https://${1}starthubs.uk${2}" middlewareType=RedirectRegex entryPointName=web routerName=http-catchall@docker middlewareName=test@docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding tracing to middleware" middlewareName=test@docker routerName=http-catchall@docker entryPointName=web
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=web middlewareName=traefik-internal-recovery
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" serviceName=whoami routerName=whoami@docker middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating load-balancer" routerName=whoami@docker entryPointName=websecure serviceName=whoami
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating server 0 http://172.21.0.3:80" routerName=whoami@docker entryPointName=websecure serviceName=whoami serverName=0
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Added outgoing tracing middleware whoami" middlewareName=tracing middlewareType=TracingForwarder entryPointName=websecure routerName=whoami@docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Added outgoing tracing middleware api@internal" middlewareType=TracingForwarder routerName=traefik@docker entryPointName=websecure middlewareName=tracing
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" middlewareType=BasicAuth entryPointName=websecure routerName=traefik@docker middlewareName=authtraefik@docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Adding tracing to middleware" entryPointName=websecure routerName=traefik@docker middlewareName=authtraefik@docker
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Creating middleware" middlewareType=Recovery entryPointName=websecure middlewareName=traefik-internal-recovery
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Try to challenge certificate for domain [starthubs.uk] found in HostSNI rule" routerName=whoami@docker providerName=leresolver.acme rule="Host(`starthubs.uk`)"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Looking for provided certificate(s) to validate [\"starthubs.uk\"]..." routerName=whoami@docker providerName=leresolver.acme rule="Host(`starthubs.uk`)"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No ACME certificate generation required for domains [\"starthubs.uk\"]." routerName=whoami@docker providerName=leresolver.acme rule="Host(`starthubs.uk`)"
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Looking for provided certificate(s) to validate [\"starthubs.uk\" \"*.starthubs.uk\"]..." providerName=leresolver.acme
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No ACME certificate generation required for domains [\"starthubs.uk\" \"*.starthubs.uk\"]." providerName=leresolver.acme
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="Looking for provided certificate(s) to validate [\"starthubs.co.uk\" \"*.starthubs.co.uk\"]..." providerName=leresolver.acme
traefik_1  | time="2020-05-24T01:44:31Z" level=debug msg="No ACME certificate generation required for domains [\"starthubs.co.uk\" \"*.starthubs.co.uk\"]." providerName=leresolver.acme
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Connection\":[\"Keep-Alive\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"5.119.95.33\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"5.119.95.33:63559\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://172.21.0.2:80" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Connection\":[\"Keep-Alive\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"5.119.95.33\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"5.119.95.33:63559\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59698\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59698\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.21.0.2:80"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59700\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59700\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.21.0.2:80"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59702\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59702\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.21.0.2:80"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59704\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://172.21.0.2:80" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59704\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59706\",\"RequestURI\":\"/\",\"TLS\":null}"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59706\",\"RequestURI\":\"/\",\"TLS\":null}" ForwardURL="http://172.21.0.2:80"
traefik_1  | time="2020-05-24T02:01:26Z" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"POST\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/1.1\",\"ProtoMajor\":1,\"ProtoMinor\":1,\"Header\":{\"Accept-Encoding\":[\"gzip,deflate\"],\"Content-Length\":[\"7449\"],\"Content-Type\":[\"application/octet-stream\"],\"Cookie\":[\"J=zHRsDovIaHCYYmEDx3JIckpZQG1m000eHaF3XsPcNnY4P0pxs2XBRw5J2H8ASnpQ9N2F7atRnHolWf0ej6CN0wphp7Xup8yOST8A+ZlVSSDIyJAtDh+/dnTONeHVuzye0excu7TlNAy5qpHM5asF96qZtdSNVunUcuMbzrWWQhXjAVkFaUoTtanhs04bAQRPt+DoWuY=\"],\"User-Agent\":[\"Apache-HttpClient/UNAVAILABLE (java 1.5)\"],\"X-Forwarded-Host\":[\"vshop.mdc.gameloft.com\"],\"X-Forwarded-Port\":[\"80\"],\"X-Forwarded-Proto\":[\"http\"],\"X-Forwarded-Server\":[\"629265f1b4ec\"],\"X-Real-Ip\":[\"172.21.0.2\"]},\"ContentLength\":7449,\"TransferEncoding\":null,\"Host\":\"vshop.mdc.gameloft.com\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"172.21.0.2:59708\",\"RequestURI\":\"/\",\"TLS\":null}"
3

Turns out this is just how Traefik handles routing when the regex can't match anything.

Since I was redirecting every http request to the server to my regex replacement, but only matching with regex those that specifically include the domain name, any requests directly to the IP address would hit the same redirect but not match the regex.

# matches only requests with the domain name
- "traefik.http.middlewares.test.redirectregex.regex=https?:\\/\\/(.*\\.)?starthubs(?:\\.co)?\\.uk(.*)?"
- "traefik.http.middlewares.test.redirectregex.replacement=https://$${1}starthubs.uk$${2}"
- "traefik.http.middlewares.test.redirectregex.permanent=true"

# sends any http request to the middleware
- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=test"

I fixed it by changing the router rule to only redirect if the domain name is present.

- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:(.+\\.)?starthubs(\\.co)?\\.uk.*}`)"