Increase maximum body size without buffering?

Traefik Traefik v2
1

Hi everyone,
behind my Traefik router I am using a Nextcloud installation. But out of the box uploading large files (like 10 GB) does not work and gets rejected by Traefik.

It can be fixed by enabling the Buffering middleware and configure the maximum allowed body size. The downside is that the whole upload request is buffered in the middleware and then when that upload finishes (from the client perspective) then it is forwarded to the backend. This can take several minutes and clients tend to timeout while waiting for the request to actually finish which is bascially the problem for me.

So the question is:

  • is it possible to increase the allowed body size without using the Buffering middleware? I guess not, right?
  • If Buffering middleware is needed to increase allowed body size: is it possible to disable the buffering in memory and instead forward it directly (streaming?) to the backend? So when in the client the upload finishes then it actually is finished in the backed?

Thank you for advice!

2 Likes
2

Hi,
i've a similar problem. But for Files above 1GB. Nextcloud is immediately rejecting the upload and throw a sabredav error. The limits in php and nextcloud are high enough, so i pretend to think traefik would be the cause.

I've set maxbodysize up to 3,3GB in traefik config but it does nothing. Do you have a working example config for traefik docker container? Is there only the label to set for this middleware or have i to declare the buffering middleware somewhere else? The traefik docs are a little bit confusing to me.

But when there would be an answer to your question, this would be my prefered solution, too.

kind regards
Markus

3

To me it seems that maxBodySize is only used for special use-case Mirroring (service).

Buffering has different parameter names. Make sure to apply the created middleware to your router.

whoami:
  #  A container that exposes an API to show its IP address
  image: traefik/whoami
  labels:
    # Create a middleware named `foo-add-prefix`
    - "traefik.http.middlewares.foo-add-prefix.addprefix.prefix=/foo"
    # Apply the middleware named `foo-add-prefix` to the router named `router1`
    - "traefik.http.routers.router1.middlewares=foo-add-prefix@docker"
4

i think it is not the buffering that is the problem but the traefik's request read timeout. fixed it for me by adding this:

    transport:
      respondingTimeouts:
        readTimeout: 0

to the entrypoint config.

1 Like
5

Made an account to ask do you have traefik v2 running or v3.1? I can't get this to work at all with v3.x whatsoever. My logs have:

2024-10-01T22:35:36-05:00 ERR github.com/vulcand/oxy/v2@v2.0.0/buffer/buffer.go:196 > vulcand/oxy/buffer: failed to read response, err: no data ready entryPointName=websecure middlewareName=nextcloud-buffering@docker middlewareType=Buffer routerName=nextcloud@docker
6

For me readTimeout fixed the issue:

  websecure:
    address: ":443"
    transport:
      respondingTimeouts:
        readTimeout: 30m

I was having issues with immich video backup a file that was about 700mb was getting stuck at 600mb and the error that printed in the log was as follows:

ERR vulcand/oxy/buffer: error when reading request body, err: context canceled entryPointName=websecure middlewareName=limit@file middlewareType=Buffer routerName=websecure-img@file
2 Likes
7

Thanks a lot! Your suggestion worked for me.