Proxy to external service (file provider http service) - Bad Gateway caused by EOF?

Traefik Traefik v2 (latest)
, ,
1

Hi There,

I'm using Traefik v2.2.1 on Docker Swarm and cloudflare DNS challenge for docker containers serving up a default HTTPS letsencrypt certificate.

All works great for the docker containers/services and they can all be accessed externally using https://SERVICE.DOMAIN but I'd also like to use Traefik as a reverse proxy for non-docker services. To test this, I created the following file provider as "traefik-custom-proxy.toml" and it's picked up by Traefik as dynamic configuration however, when I try to navigate to https://TEST.DOMAIN externally, rather than this being proxied to the local LAN address of http://192.168.1.2:80, I get the following error in the traefik.log debug?

Any ideas what I might be doing wrong here?

time="2020-06-13T01:22:48+08:00" level=debug msg="Serving default certificate for request: \"\""
time="2020-06-13T01:22:50+08:00" level=debug msg="vulcand/oxy/roundrobin/rr: begin ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.9,es;q=0.8,ms;q=0.7,ko;q=0.6\"],\"Cache-Control\":[\"max-age=0\"],\"Cookie\":[\"_ga=GA1.2.1642069948.1586136495; _gid=GA1.2.984113571.1591960016\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36\"],\"X-Forwarded-Host\":[\"TEST.<DOMAIN>\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"2ad04db70f65\"],\"X-Real-Ip\":[\"<REMOTE_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"TEST.<DOMAIN>\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<REMOTE_ADDRESS>:50505\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2020-06-13T01:22:50+08:00" level=debug msg="vulcand/oxy/roundrobin/rr: Forwarding this request to URL" ForwardURL="http://192.168.1.2:80" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.9,es;q=0.8,ms;q=0.7,ko;q=0.6\"],\"Cache-Control\":[\"max-age=0\"],\"Cookie\":[\"_ga=GA1.2.1642069948.1586136495; _gid=GA1.2.984113571.1591960016\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36\"],\"X-Forwarded-Host\":[\"TEST.<DOMAIN>\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"2ad04db70f65\"],\"X-Real-Ip\":[\"<REMOTE_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"TEST.<DOMAIN>\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<REMOTE_ADDRESS>:50505\",\"RequestURI\":\"/\",\"TLS\":null}"
time="2020-06-13T01:22:50+08:00" level=debug msg="'502 Bad Gateway' caused by: EOF"
time="2020-06-13T01:22:50+08:00" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.9,es;q=0.8,ms;q=0.7,ko;q=0.6\"],\"Cache-Control\":[\"max-age=0\"],\"Cookie\":[\"_ga=GA1.2.1642069948.1586136495; _gid=GA1.2.984113571.1591960016\"],\"Sec-Fetch-Dest\":[\"document\"],\"Sec-Fetch-Mode\":[\"navigate\"],\"Sec-Fetch-Site\":[\"none\"],\"Sec-Fetch-User\":[\"?1\"],\"Upgrade-Insecure-Requests\":[\"1\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36\"],\"X-Forwarded-Host\":[\"TEST.<DOMAIN>\"],\"X-Forwarded-Port\":[\"443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"2ad04db70f65\"],\"X-Real-Ip\":[\"<REMOTE_ADDRESS>\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"TEST.<DOMAIN>\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"<REMOTE_ADDRESS>:50505\",\"RequestURI\":\"/\",\"TLS\":null}"

Below is my dynamic "traefik-custom-proxy.toml"

[http.routers]
  [http.routers.testhttp-rtr]
     entryPoints = ["https"]
     rule = "Host(`TEST.<DOMAIN>`)"
     service = "testhttp-svc"
     middlewares = ["secure-chain"]
     [http.routers.testhttp-rtr.tls]
       certresolver = "letsencryptresolver"

[http.services]
  [http.services.testhttp-svc]
    [http.services.testhttp-svc.loadBalancer]
      passHostHeader = true
      [[http.services.testhttp-svc.loadBalancer.servers]]
        url = "http://192.168.1.2:80"

The following is my "dynamic.toml" file which is also picked up by Traefik as dynamic config:

# dynamic.toml

[http.middlewares]

  [http.middlewares.secure-chain.chain]
    middlewares = [ "rate-limit", "secure-headers" ]

  [http.middlewares.rate-limit]
    [http.middlewares.rate-limit.rateLimit]
      average = 100
      burst = 50

  [http.middlewares.secure-headers.headers]
    AccessControlAllowMethods = ["GET", "OPTIONS", "PUT"]
    #AccessControlAllowOrigin = "origin-list-or-null"
    AccessControlMaxAge = 100
    #AddVaryHeader = true
    BrowserXssFilter = true
    ContentTypeNosniff = true
    ForceSTSHeader = true
    FrameDeny = true
    SSLRedirect = true
    #STSIncludeSubdomains = true
    STSPreload = true
    CustomFrameOptionsValue = "SAMEORIGIN"
    ReferrerPolicy = "same-origin"
    FeaturePolicy = "geolocation 'self'"
    STSSeconds = 315360000

    [http.middlewares.secure-headers.headers.customResponseHeaders]
      #X-Robots-Tag = "none,noindex,nofollow,noarchive,nosnippet,notranslate,noimageindex,"
      X-Robots-Tag = "noindex,nofollow,noarchive,nosnippet,notranslate,noimageindex,"


[tls]
  [tls.options]
    [tls.options.default]
       minVersion = "VersionTLS12"
       #sniStrict = true
       cipherSuites = [
         "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
         "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
         "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
         "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
         "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
         "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
         "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
         "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305",
         "TLS_AES_128_GCM_SHA256",
         "TLS_AES_256_GCM_SHA384",
         "TLS_CHACHA20_POLY1305_SHA256",
         "TLS_FALLBACK_SCSV",
      ]

Or if anyone has a sample working Traefik v2.x swarm configuration that supports LE, Docker and File Providers, I’d be very appreciative to take a look!

Thanks for your support!