Chunk size in multipart request

[fuch1m]

Summary

Hi,

I know that I can set the overall body limit with

.layer(DefaultBodyLimit::max(1024 * 1024 * 1024))

but what is the size of the individual chunks received? I couldn't find that in code and would like to know what is the default chunk size and if we can change/limit it?

I searched and found this: #1682
But I don't know if this also applies here and maybe someone can explain why this is a job for hyper.

Thank you

axum version

0.7.5

[mladedav]

Are you asking to receive each chunk individually to know its size or to configure something to reject requests that send too large chunks?

From the linked question I assume it's the latter?

As far as I know there is no way in HTTP to declare maximum chunk size so the framework has to solve it itself and it could be surprising to some clients.

However, multiple chunks can be inflight at the same time and hyper might be buffering data in background so if you're trying to cap data usage per request (as the linked question tries to), I'm not sure this would work.

I am not sure if the stream guarantees you'll get the data on chunk boundaries.

So what are you trying to do with the chunks?

[fuch1m]

Thank you for your answer. I use multipart/form-data to upload a single large (up to 1GB) file. I run on a rather resource constrained system (CPU/RAM) and try to set RAM limits in docker-compose.yml and wanted to know what max size of chunks I could expect, because I don't want my application to fail because I set a limit of 10MB RAM and then I receive a 15MB chunk and fail. The received chunks are directly appended to a file on disk.

If have written before an application using libcurl and there is an option to set the chunk size (https://curl.se/libcurl/c/CURLOPT_BUFFERSIZE.html) which might result in more but smaller chunks. I search for a similar setting in axum.

[mladedav]

When you talk about chunks, you don't meant chunked transfer encoding, you just generally mean part of the received data, right?

In that case it really needs to be configured on hyper with something like this.

I don't think there's currently an easy way to configure this while using Axum.

[fuch1m]

I use this, so I think that is not chunked transfer encoding but just receiving parts of the uploaded data. And I guess if I don't change anything this would default to 400KB buffer per connection as stated in your link. So if I allow uploading of 3 files simulatenously it would require 3*400KB=1,2MB RAM for the buffers required for these chunks.