[Feature: Multipart-parser] Add native ReadableStream support for multipart parts with breaking change considerations

[Nyoxis]

Overview

I have implemented native ReadableStream support for multipart parts aiming to improve streaming efficiency and compatibility with modern standard streams. This proposal discusses the design and invites feedback from maintainers and the community.

• My PR: feat(multipart-parser): add ReadableStream support for multipart parts #10782
• Existing multipart streaming PR: feat(multipart-parser): add streaming support #10764

Context and Motivation

Currently, the existing PR for streaming parts uses a state change callbacks mechanism that returns chunks of parts rather than native ReadableStreams. My implementation differs by exposing ReadableStream instances per multipart part, which aligns better with the Fetch API and modern streaming patterns.

I am unsure why originally wasn't implemented ReadableStreams for multipart parts yet — there might be considerations, performance or compatibility trade-offs I might have missed.

Additionally, I am uncertain if the parseMultipart in the library is intended to yield non-streamed (fully buffered) parts, or if that should be adjusted to keep consistency with the streaming API. I was unable to find any existing implementation clarifying this point.

Breaking Changes

My changes introduce breaking API changes that could affect existing implementations. This discussion aims to gather feedback on whether this direction is acceptable and under what conditions.

Discussion Points

• Pros and cons of state callback chunks vs. ReadableStream per part approach
• Consistency between streamed and non-streamed parsers’ outputs
• Potential impact on users and migration paths
• Performance and resource usage implications

I welcome all feedback, suggestions, and alternative ideas to refine this design and reach consensus.

[ahuaiqom]

Hi @Nyoxis — great proposal 👏

Exposing native ReadableStream per part is the right direction (closer to Fetch/Web Streams, better back-pressure & memory). I’d suggest shipping it behind a new API with a clear migration path, plus an adapter so existing callback consumers don’t break.

✅ API shape (suggestion)

Keep the current callback API as-is, and add a streaming-first variant that returns an async iterator of parts with a Web ReadableStream body:

// New, non-breaking API
type MultipartPart = {
name: string;
filename?: string;
contentType?: string;
headers: Headers;
body: ReadableStream; // Web stream
};

declare function parseMultipartStream(
req: Request, // standard Fetch Request
opts?: { maxParts?: number; maxPartSize?: number }
): AsyncIterable;

Consumer examples

Edge / Fetch runtimes

for await (const part of await parseMultipartStream(request)) {
if (part.filename) {
// stream to storage
await someBlobStore.put(part.filename, part.body);
} else {
const text = await new Response(part.body).text();
// handle field value
}
}

Node adapter (when a Node stream is needed)

import { readableWebToNode } from 'stream/web';
const nodeStream = readableWebToNode(part.body);
nodeStream.pipe(fs.createWriteStream(dest));

Back-compat adapter (emulate old “chunk callback”)

async function onEachChunk(part: MultipartPart, cb: (c: Uint8Array)=>void) {
const reader = part.body.getReader();
for (;;) {
const { value, done } = await reader.read();
if (done) break;
cb(value);
}
}

🔁 Migration & breaking-change handling

Ship as new function (parseMultipartStream) and deprecate the callback style over time; keep it working via the adapter above.

Provide a codemod (or docs snippet) that converts onChunk(part, buf) to for await ... over ReadableStream.

Version gates: mark the old API “stable” this release; announce deprecation window before removal.

🧪 Performance & correctness checklist

Back-pressure: verify slow consumer doesn’t buffer the whole file (compare RSS vs callback API).

Many small parts vs few large parts benchmarks; include GC & throughput.

Edge constraints: no Node-only types; ensure it runs in Workers/Deno runtimes.

Security limits: maxParts, maxPartSize, maxHeaderSize, early abort on boundary abuse.

Error semantics: consistent cancel/abort when consumer throws; ensure upstream stream is closed.

Interoperability: example showing streaming directly to fetch()/S3 SDKs that accept Web streams.

✍️ Docs

Position the old API as “callback (legacy)” and the new one as “ReadableStream (recommended)”.

Provide end-to-end examples for file upload → storage and form fields without buffering.