Backport WHATWG URL to 6.x #17365

TimothyGu · 2017-11-28T05:39:53Z

Includes all commits related to the WHATWG URL parser up until now. To ease backporting, many changes to url documentation not directly related to the new parser are also included. For compatibility concerns, the new url.format(whatwgURLObj) overload and WHATWG URL support in fs and http are not backported.

PRs #11464, #9246, #13362, which the current WHATWG URL implementation requires, are backported in separate commits as they are logically separate, even if the commits themselves were made after the WHATWG parser landed and contained changes to it.

Fixes: nodejs/Release#208

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

url

- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer PR-URL: nodejs#11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: nodejs#9246 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Roman Reiss <me@silverwind.io> Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. PR-URL: nodejs#13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

Implements WHATWG URL support. Example: ``` var u = new url.URL('http://example.org'); ``` Many, many other commits improving the implementation have been squashed into this backport PR. They are not listed separately here for brevity. PR-URL: nodejs#7448 Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>

MylesBorins · 2017-11-28T05:45:53Z

Wow!!!

@nodejs/LTS any concerns around this? We will likely have to wait until the next minor release for this

jasnell

Very nice.

TimothyGu · 2017-11-30T08:00:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/11815/

joyeecheung

LGTM with a green CI

MylesBorins · 2017-11-30T12:52:26Z

Please remember that this should not land on the staging branch until we are ready to do a minor release

…

On Nov 30, 2017 7:03 PM, "Joyee Cheung" ***@***.***> wrote: ***@***.**** approved this pull request. LGTM with a green CI — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17365 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecV0IwOAHJvdDeSeEnZDtl3K0EmueMks5s7n1pgaJpZM4Qst82> .

MylesBorins · 2018-01-17T06:51:49Z

one more ci: https://ci.nodejs.org/job/node-test-pull-request/12575/

TimothyGu · 2018-01-17T10:44:06Z

Looks like either rebasing errors (https://ci.nodejs.org/job/node-test-linter/15225/console) or infra issues (https://ci.nodejs.org/job/node-test-commit-linux/nodes=alpine35-container-x64/15605/console). Maybe something else too as https://ci.nodejs.org/job/node-test-pull-request/12575/ shows "aborted".

MylesBorins · 2018-01-17T20:38:17Z

One more CI: https://ci.nodejs.org/job/node-test-pull-request/12588/

- Add IsInvalidated() method - Add capacity() method for finding out the actual capacity, not the current size, of the buffer - Make IsAllocated() work for invalidated buffers - Allow multiple calls to AllocateSufficientStorage() and Invalidate() - Assert buffer is malloc'd in Release() - Assert buffer has not been invalidated in AllocateSufficientStorage() - Add more descriptive comments describing the purpose of the methods - Add cctest for MaybeStackBuffer Backport-PR-URL: #17365 PR-URL: #11464 Reviewed-By: Steven R Loomis <srloomis@us.ibm.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com>

MylesBorins · 2018-01-18T02:58:10Z

ARM failures are unrelated

landed in 3c4bb3c...3639d0c

Backport-PR-URL: #17365 PR-URL: #9246 Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com> Reviewed-By: Roman Reiss <me@silverwind.io> Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Remove custom tests for invalid IDNA domains in url-idna.js in favor of the more comprehensive official set. Backport-PR-URL: #17365 PR-URL: #13362 Refs: whatwg/url#309 Refs: web-platform-tests/wpt#5976 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>

Implements WHATWG URL support. Example: ``` var u = new url.URL('http://example.org'); ``` Many, many other commits improving the implementation have been squashed into this backport PR. They are not listed separately here for brevity. Backport-PR-URL: #17365 PR-URL: #7448 Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>