Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content is corrupted in case of utf8 multibyte characters #743

Closed
Biktop opened this issue Nov 15, 2019 · 5 comments
Closed

Content is corrupted in case of utf8 multibyte characters #743

Biktop opened this issue Nov 15, 2019 · 5 comments

Comments

@Biktop
Copy link

Biktop commented Nov 15, 2019

I found that in version 5.1.0 downloaded cvs parsed incorrectly in case if source data contains utf8 multibyte characters (whereas 5.0.2 didn't have this issue). I noticed that PapaParse tried to download second chunk even if there no any extra bytes:

First chunk:

< Content-Length: 184027
< Content-Range: bytes 0-184026/184027

Second chunk (not necessary)

> Range: bytes=183546-5426425
< Content-Length: 481
< Content-Range: bytes 183546-184026/184027

After some digging I think that problem here
this._start += xhr.responseText.length;

Because Content-Length can be greater then text length (because some characters can be 2 or more bytes).

I think it should be something like this:
this._start += +xhr.getResponseHeader('Content-Length');

@pokoli
Copy link
Collaborator

pokoli commented Nov 19, 2019

I think this should be fixed by #745 could you please test it?

@Biktop
Copy link
Author

Biktop commented Nov 19, 2019

I've tested it and it seems that it works!

pokoli pushed a commit that referenced this issue Nov 20, 2019
@pokoli
Copy link
Collaborator

pokoli commented Nov 20, 2019

Ok, so I've merged #745 which fixes this

@pokoli pokoli closed this as completed Nov 20, 2019
@Biktop
Copy link
Author

Biktop commented Dec 17, 2019

Are going to publish this fix soon?

@pokoli
Copy link
Collaborator

pokoli commented Dec 18, 2019

I've just published 5.1.1 which contains this fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants