Chunked File Transfer System with Configurable Partition Size

[mthsmb]

Is there an existing feature request for this?

• I have searched the existing issues before opening this feature request.

Describe the feature you would like to see.

Currently, large file transfers fail when using Cloudflare Tunnels due to the 100MB upload limit, as well as other limitations shown in issue #4696. Key problems include:

1. Cloudflare Tunnels 100MB upload limit restriction
2. Transfer timeouts (occurring ~2 minutes)
3. Nginx reverse proxy limitations
4. Complete transfer failure without resume capability
5. No partial transfer recovery options

This enhancement would implement a chunked file transfer system specifically designed to work within these limitations.

Describe the solution you'd like.

Implement a chunked file transfer system that respects Cloudflare Tunnels limitations:

1. Chunk Management:

• Default chunk size of 95MB (staying under Cloudflare Tunnels 100MB limit)
• Configurable chunk size with maximum of 95MB when Cloudflare is detected
• Automatic file splitting and reassembly
• Chunk verification on completion

2. Configuration Options:

• Adjustable chunk size (with enforced 95MB cap for Cloudflare)
• Configurable concurrent chunk transfers
• Retry settings per chunk
• Timeout settings per chunk operation

3. Recovery System:

• Independent chunk retry mechanism
• Transfer resume from last successful chunk
• Chunk-level integrity verification
• Failed chunk logging and reporting

4. User Interface:

• Clear progress tracking per chunk
• Overall transfer progress display
• Pause/Resume functionality
• Error reporting with chunk-specific details

Additional context to this request.

This directly addresses the issue #4696 while specifically considering Cloudflare Tunnels limitations:

• Enforces chunk sizes compatible with 100MB Cloudflare limit
• Handles timeouts by breaking transfers into smaller operations
• Provides resilience against connection issues
• Enables reliable large file transfers even with infrastructure limitations

Implementation considerations:

• Automatic Cloudflare detection for appropriate chunk sizing
• Efficient chunk management to minimize overhead
• Clear error handling and reporting
• Progress persistence for long-running transfers