Can you ignore invalid Content-Length on the response from a backend instead of returning 500?

Traefik Traefik v2
1

Hello, I'm struggling with some software of questionable quality (UniFi Controller) which generally works behind Traefik, but when you try to download a backup file the response from the backend has the Content-Length header set to some apparently random negative value. This obviously violates the spec, content length if present must be a positive integer or 0, but as usual browsers don't enforce that and the download goes through albeit with no progress bar. Traefik however does what is probably the right thing and throws 500 Internal Server Error.
Sample debug log:

reverse-proxy_1 | time="2021-07-16T14:17:29Z" level=debug msg="'500 Internal Server Error' caused by: net/http: HTTP/1.x transport connection broken: bad Content-Length \"-1963422688\""

reverse-proxy_1 | time="2021-07-16T14:17:29Z" level=debug msg="vulcand/oxy/roundrobin/rr: completed ServeHttp on request" Request="{\"Method\":\"GET\",\"URL\":{\"Scheme\":\"\",\"Opaque\":\"\",\"User\":null,\"Host\":\"\",\"Path\":\"/dl/autobackup/autobackup_6.2.26_20210716_0500_1626411600025.unf\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\",\"RawFragment\":\"\"},\"Proto\":\"HTTP/2.0\",\"ProtoMajor\":2,\"ProtoMinor\":0,\"Header\":{\"Accept\":[\"*/*\"],\"Accept-Encoding\":[\"gzip, deflate, br\"],\"Accept-Language\":[\"en-US,en;q=0.5\"],\"Cookie\":[\"unifises=[REDACTED]; csrf_token=[REDACTED]\"],\"Referer\":[\"https://unifi.[REDACTED].com:8443/manage/site/givqab4c/settings/backup\"],\"Te\":[\"trailers\"],\"User-Agent\":[\"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0\"],\"X-Forwarded-Host\":[\"unifi.[REDACTED].com:8443\"],\"X-Forwarded-Port\":[\"8443\"],\"X-Forwarded-Proto\":[\"https\"],\"X-Forwarded-Server\":[\"4153a08587b0\"],\"X-Real-Ip\":[\"10.42.10.1\"]},\"ContentLength\":0,\"TransferEncoding\":null,\"Host\":\"unifi.[REDACTED].com:8443\",\"Form\":null,\"PostForm\":null,\"MultipartForm\":null,\"Trailer\":null,\"RemoteAddr\":\"10.42.10.1:40423\",\"RequestURI\":\"/dl/autobackup/autobackup_6.2.26_20210716_0500_1626411600025.unf\",\"TLS\":null}"

So what I need is to have Traefik ignore the invalid negative Content-Length and either pass it through or remove the header as it's not required as far as I can tell. Also submit a bug to Ubiquity but I have no faith that it will be fixed within the next few years. Thanks.

2

Hello @workalt23184,

Your predicament is understandable, however allowing or ignoring invalid content-length headers opens Traefik up to a whole range of attacks, and exposes a variety of security issues.

Furthermore, Traefik uses the standard Go reverse proxy code to ensure that upstream fixes are present in our codebase. This means that we can't easily implement this without creating our own Go fork.

Another option would be to use a stateless proxy like squid or varnish to modify the raw headers of your response in cases where a backup is created. This may allow Traefik to properly front your service without having to deal with invalid backend responses.

3

Thanks for your response @daniel.tomcej so am I correct that if a proxy like how you mentioned could strip the content-length header that should be enough? These files are 1-3 GB, so I'd need to do this with something that works in stream mode. As far as security I see request smuggling as the main issue which would involve invalid content length headers from a client to the server. Would there be security issues with stripping the content length on these responses? Sorry if this is getting off topic now. It would probably be easier to just copy the files over SFTP honestly..

4

Hello @workalt23184,

It should be enough to strip the content-length, although you may have to test to confirm. Proxies like varnish can strip and automatically recalculate and add the proper content-length for you.

It should be fine in your instance, as you are providing the source content, and managing the proxy, so at the end of the chain it should be fine. You aren't going to send malicious traffic smuggled out (hopefully).