url, querystring: web platform tests broken by WHATWG URL implementation

[joyeecheung]

Version: 90c2ac7

Broken tests discovered in #11079 that don't have tracking issues:

Leading ??

In upstream urltestdata.json:

  {
    "input": "??a=b&c=d",
    "base": "http://example.org/foo/bar",
    "href": "http://example.org/foo/bar??a=b&c=d",
    "origin": "http://example.org",
    "protocol": "http:",
    "username": "",
    "password": "",
    "host": "example.org",
    "hostname": "example.org",
    "port": "",
    "pathname": "/foo/bar",
    "search": "??a=b&c=d",
    "searchParams": "%3Fa=b&c=d",
    "hash": ""
  }

Expected searchParams.toString() to be %3Fa=b&c=d, got a=b&c=d(the second ? is ignored).

Also presents in upstream url-constructor.html

var url2 = bURL('http://example.org/file??a=b&c=d')
assert_equals(url2.search, '??a=b&c=d')
assert_equals(url2.searchParams.toString(), '%3Fa=b&c=d')

url2.href = 'http://example.org/file??a=b'
assert_equals(url2.search, '??a=b')
assert_equals(url2.searchParams.toString(), '%3Fa=b')

Note: this only appears when we parse through URL

new URL('http://example.org/file??a=b&c=d').searchParams.toString()
// 'a=b&c=d'
new URLSearchParams('??a=b&c=d').toString()
// '%3Fa=b&c=d'

Space should be escaped as +

In upstream url-constructor.html

searchParams.append('i', ' j ')
assert_equals(url.search, '?e=f&g=h&i=+j+')
assert_equals(url.searchParams.toString(), 'e=f&g=h&i=+j+')
assert_equals(searchParams.get('i'), ' j ')

In urlsearchparams-stringifier.html

var params = new URLSearchParams();
params.append('a', 'b c');
assert_equals(params + '', 'a=b+c');
params.delete('a');
params.append('a b', 'c');
assert_equals(params + '', 'a+b=c');

Currently it's escaped as %20 by the querystring module. I suspect fixing this in querystring might be too breaking though.

[mscdex]

I'm starting to wonder if the whatwg url implementation should just have its own querystring implementation ...

[joyeecheung]

Yeah I agree, when it is the spec behavior that is breaking(and breaking big), it's impossible to make a choice. I remember @TimothyGu mentioned he had implemented another one in C++?

[TimothyGu]

I'm starting to wonder if the whatwg url implementation should just have its own querystring implementation

That was what I was pushing for. See #10967 (comment) and #10821.

[jasnell]

Yes, my intent all along was to have a separate parsing algorithm for querystring but just hadn't managed to get to it.

[TimothyGu]

Actually, reopening this since only first issue of leading ?? is fixed with fa41dd1.

[joyeecheung]

We can probably close this in favor of #10821?