Expose origins as ASCII or Unicode

[annevk]

In particular, it seems browsers are not interoperable for url.origin, aOrArea.origin, self.origin, location.origin, location.ancestorOrigins, and messageEvent.origin when it comes to this. I can write some tests and file bugs.

[bzbarsky]

WebKit/Blink simply have no code at all that can do https://html.spec.whatwg.org/multipage/browsers.html#unicode-serialisation-of-an-origin right now, so any time they serialize an origin they always do https://html.spec.whatwg.org/multipage/browsers.html#ascii-serialisation-of-an-origin no matter what the spec says.

We definitely need tests around this stuff.

@mikewest

[annevk]

Even for postMessage()? How is anyone writing reliable code for that then? Implement their own ToASCII and ToUnicode?

[bzbarsky]

Even for postMessage()

You mean in terms of the .origin of the MessageEvent? It's punycode in Chrome and Safari.

How is anyone writing reliable code for that then?

As long as you only expect messages from your own domain, you could compare to location.origin or something and it would work across all browsers.

If you expect messages from other hostnames, and they're IDN, then I expect people either don't check .origin or end up with code that only works in some browsers. I'm not sure which is more likely.

[annevk]

I'm really surprised this is not interoperable. I vaguely recall Adam Barth and Ian Hickson specifically agreeing on this design. I'm probably remembering it incorrectly or maybe Adam or someone else changed their mind.

[annevk]

@sleevi I now understand why in whatwg/url#274 you were concerned about exposing ToUnicode to script. It would actually be the first time for Chrome.

[annevk]

I was going to write tests, but if Chrome and Safari indeed never expose Unicode origins I'm not sure what the point is since I don't know what the correct results should be.

@foolip who can make a decision here on behalf of Chrome? @hober who can make a decision here on behalf of Safari? @travisleithead who can make a decision on behalf of Edge?

It seems pretty clear Firefox would favor ToUnicode, but if Firefox is alone that's not a tenable position. Though I do wonder what Firefox returns when mixed scripts are used within the same label. Does it return what the UI would display in that case (which I believe is Punycode) or what ToUnicode as defined at http://www.unicode.org/reports/tr46/#ToUnicode returns (which I believe is not Punycode)? If it's not always the latter than the Firefox solution seems a lot harder to get interoperable since we cannot really require UI even though it would be good here...

[zcorpan]

SELECT * FROM (
SELECT page, url, REGEXP_EXTRACT(body, r'(.{50}\.origin\s*!?===?\s*["\'][^\)]+)') AS match
FROM [httparchive:har.2017_03_15_chrome_requests_bodies]
) WHERE match != "null"
AND match CONTAINS "xn--"
OR REGEXP_MATCH(match, r'[\x80-\xff]')

This only brought up a single result, which was a false positive:

	 
1	http://www.alienvault.com/	https://www.alienvault.com/cache/98e1630ea768312e0375124c2f1cf8fc795204c6.1489172110.js	event){��if($('.vidyard_iframe').length&&event.origin==='https://play.vidyard.com'&&!setupVidyardDone

Searching github reveals existence proof of code expecting punycode (also in Origin request header?). I don't know if it's possible to search github for things expecting Unicode serialization...

https://github.com/search?utf8=✓&q="origin+https+xn"+OR+"origin+http+xn"&type=Code

[annevk]

The Origin request header must use ToASCII per standard as sticking UTF-8 bytes in HTTP header values would be a little weird (though not unheard of). Not sure if that's tested but I suspect that's fully interoperable, but after this episode I probably can't be trusted.