504 Timeout after 60 seconds in large file uploads (No, its not readTimeout and its also not the wrong network)

Traefik Traefik v3 (latest)
,
1

Hey y'all. I am selfhosting immich, and I have a weird problem with that it fails to upload large files. My architecture is: 1 public VPS hosting traefik with a file provider, and one private VPS hosting immich. Both are connected via a tailscale tunnel. This works wonderfully when browsing normally and uploading pictures. But as I said it fails for large files

At first it failed with 499 Errors.

github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:117 > 499 Client Closed Request error="context canceled"
Feb 04 09:45:51 oracle1 traefik[36124]: 80.187.82.174 - - [04/Feb/2025:09:44:51 +0000] "POST /api/assets HTTP/1.1" 499 21 "-" "-" 3 "photos-router@file" "http://immich:2283" 59999ms

Found this Discussion and implemented the fix by setting the corresponding timeouts which were not zero by default to zero.

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
          permanent: true
    transport:
      respondingTimeouts:
        readTimeout: 0
        idleTimeout: 0
      #middlewares:
        #- traefik-crowdsec-bouncer@file
  websecure:
    address: ":443"
    #http:
      #middlewares:
        #- traefik-crowdsec-bouncer@file

I am now getting a the 504 Gateway Timeout Error with the same duration:

Feb 04 10:35:15 oracle1 traefik[36657]: 2025-02-04T10:35:15Z DBG github.com/traefik/traefik/v3/pkg/proxy/httputil/proxy.go:117 > 504 Gateway Timeout error="read tcp 10.0.0.229:443->80.187.82.174:14540: i/o timeout"
Feb 04 10:35:15 oracle1 traefik[36657]: 80.187.82.174 - - [04/Feb/2025:10:34:15 +0000] "POST /api/assets HTTP/1.1" 504 15 "-" "-" 9 "photos-router@file" "http://immich:2283" 60282ms

Is it possible that there is a timeout after which the service must respond? Because I really do not understand how this is possible. There is no second reverse proxy in front of immich

2

Check Traefik static reference and search for further "timeout" options.

3

Ugh, so the problem was that I forgot to add the Timeouts to Websecure. They were only applied to the Web Entrypoint. Obvious in retrospekt, but well...