Binary body of a request gets corrupted in the handler

[Ovaron1994]

Environment

Name	Version
msw	0.35.0
browser	Microsoft Edge
OS	Windows 10

Request handlers

Mocking setup via storybook add-on:

import { rest } from "msw";

component.parameters = {
  msw: [
    rest.post(
      "/api/xyz/42",
      ( req,res, ctx ) => { ... }]];

In the req.body I expect a multipart content which owns an jpeg image.

In preview.js I have:

import { initializeWorker, mswDecorator } from "msw-storybook-addon";

initializeWorker();
addDecorator(mswDecorator);

Actual request

The request is a multipart content which contains a jpeg image sent by fetch.

Current behavior

The data received in the request handler is a damaged jpeg file. After a lot debugging I found out that the image data gets damaged at two locations:

1. In mockServiceWorker.js function getResponse. There the body of the request is extracted by using request.text().
   This damages the binary data because the data is converted to utf-8.

msw/src/mockServiceWorker.js

Line 175 in 11bb244

const body = await request.text()

The request body is always decoded using UTF-8

2. In RequestHandler.ts function parseMultipartData there is a call to the constructor of File with

msw/src/utils/internal/parseMultipartData.ts

Line 87 in 11bb244

: new File([contentBody], filename, { type: contentType })

This fails because the content body given to the File constructor is encoded as UTF-8 but here the data have to be given as an ArrayBuffer.

Expected behavior

Expected is that the data from the post request will not get damaged.
To prevent a damage of the image for 1. the content should be get as unicode string by using something like this:

const body = String.fromCharCode(...new Uint8Array(await request.arrayBuffer()))

For 2. we need transform the extracted data of the multipart data so the image correctly used for the creation of the File.
To transform it back from unicode string to ArrayBuffer we need soething like this:

const encoded = Uint8Array.from([...contentBody].map(ch => ch.charCodeAt())).buffer;
const imageFile = new File(encoded, filename, { type: contentType });

[Ovaron1994]

Found the solution.

In the file mockServiceWorker.js replace

const body = await request.text()

with

const unicodeBody = String.fromCharCode(...new Uint8Array(await request.arrayBuffer()));
const body = unescape(encodeURIComponent(unEncodedBody));

Now the body is an encoded utf-8 string.

The following call to the File constructor in RequestHandler.ts at function parseMultipartData uses the utf-8 encoded string
and the data is received without loss in the request handler function within your test/stroy.

[kettanaito]

Hey, @Ovaron1994. Thank you for reporting this. I'd like to clarify a few things before we can proceed with discussing a solution to this:

1. Does the issue happen only for multipart binary response bodies?
2. Can you confirm that a non-multipart (regular) binary response body is sent not corrupted?
3. Can you tell what's the difference between your use case and our binary response body integration test? Our current test doesn't catch this issue so I'd like to have another test so we could reproduce it reliably.

Ideally, if you could put your usage scenario into an integration test in our repository that'd speed up the work on this issue tremendously. Please, if you have some time, consider doing so. You can learn how to contribute with a test, as well as browser the existing tests for practical reference. Thank you.

[Ovaron1994]

Hi kettanaito,

1. The issue did not happen for binary response bodies. It appears for binary request bodies. I want to analyze/look in the data sent via POST request.
2. Have to try to send a POST request with binary data in non-multipart way.
3. Have to write a test which send a part of binary data, for example the first 10 bytes of an jpeg image, to show that these bytes are received damaged in the mocked request handler.

[Ovaron1994]

Hi @kettanaito,

need help here, Added a test but I don't get the test to run as expected.
Added body-form-binary-data.node.test.ts but it seems that the request is not handled as on browser.
The multipart request is not handled by getResponse(...) and parsemultipart(...).
I get it not run in a debugger so I could not the call stack.

Michael

[PupoSDC]

Hello!

Im suffering with the same issue. I tried the proposed solution, but due to the size of the images im trying to upload I hit a Maximum call stack exceeded for String::fromCharCode.

[Ovaron1994]

Hi PupoSDC,

have you tried to use the fromCharCode for single characters in a for loop like this:

let body = "";
const uint8Arr = new Uint8Array(await request.arrayBuffer());
for (let i = 0; i < uint8Arr.byteLength; i++) {
body += String.fromCharCode(uint8Arr[i])
}

This is not an effective way but it should work.

Another way should be:

const body = new TextDecoder().decode(new Uint8Array(await request.arrayBuffer()));

Did it work with this?

Greetings Michael

[PupoSDC]

I had tried the one using TextDecoder but that one failed due to "latin1" characters. I need to give your first option a try.

In the end i decided to go a different route. In my scenario i was trying to mock a image upload / download flow. I opted to just ignore the blob that is sent on the upload, and just returning a mock image on the download. It looks a bit awkward, but its acceptable for a mock.

[kettanaito]

Released: v0.43.0 🎉

This has been released in v0.43.0!

• 📄 Release notes
• 📦 npm package

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.

---

Predictable release automation by @ossjs/release.

[nknapp]

@kettanaito I don't think that this issue is resolved in v0.43.0

I am still getting it with 0.44.2 when trying to make PATCH request with binary request bodies and as @Ovaron1994 stated, the const body = await request.text() is the place where data gets corrupted.

Maybe you can re-open this issue.

[nknapp]

It seems works in "fallback mode" btw. It's a bit difficult to say, because the only way to get into fallback mode was to set a breakpoint here and set "useFallbackMode" to "true" in the debugger.

Maybe a quick solution to this problem is to allow to set fallback mode manually.

[kettanaito]

Hey, @nknapp. I'd encourage you to try the latest version of the library. 0.43 was a quite long time ago. Perhaps the issue has been resolved since then. If it's not, we're likely to tackle it as a part of #1404.

[theishinz-maersk]

Hey. I'm still experiencing this issue with latest version of the MSW.
The mockServiceWoerker.js is called request.text() despite it being a blob. I would expect a request.blob() beeing called.
https://github.com/mswjs/msw/blob/main/src/mockServiceWorker.js#L250

Code for the request:

export function methodName(
  document: File,
): Promise<AxiosResponse<...>> {
  const formData = new FormData();
  formData.append("data", new Blob([JSON.stringify(data)], { type: "application/json" }));
  return axios.post(`url`, formData, {
    headers: {
      "Content-Type": "multipart/form-data",
    },
  });
}

[kettanaito]

Hi, @theishinz-maersk. I'm sorry to hear you're experiencing issues regarding this.

You're right, we shouldn't read the request body as text all the time. We've already migrated away from it and now reading request body as ArrayBuffer, which will work for your use case as well:

msw/src/mockServiceWorker.js

Line 256 in ccc576d

body: requestBuffer,

That being said, I'm not going to backport this fix to the current version of MSW due to time considerations. I highly recommend you migrate your app to the msw@next version by following this #1464. Thanks for understanding.