gh-48241: Clarify URL needs to be encoded when provided to urlopen and Request

[mblahay]

Adding note about url needing to be encoded when provided to the urlopen function as well as the Request class.

This is a documentation change

• Issue: urllib.request.urlopen does not handle non-ASCII characters #48241

[ghost]

All commit authors signed the Contributor License Agreement.

[mblahay]

@ambv This is ready

[ambv]

Do you think we can drop the comma between "encoded" and "URL"?

[miss-islington]

Thanks @mblahay for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

[bedevere-bot]

GH-103891 is a backport of this pull request to the 3.11 branch.

[vadimkantorov]

A bit hard to understand what means Open url, which can be either a string containing a valid, properly encoded UR and the notions valid, properly encoded. If it accepts a string, there probably need to be some RFC reference or examples of what's properly encoded and how to encode sth properly.

My problem: is urllib.request.urlopen('https://google.com/?q=…') fails with encoding error as does urllib.request.urlopen(urllib.parse.quote('https://google.com/?q=…',safe=':/')) fails with not found error as quote also corrupts the question mark: https://google.com/%3Fq%3D%E2%80%A6'

I've tried

urlparsed = urllib.parse.urlparse(url)
urlparsed_unicode_sanitized_query = urllib.parse.ParseResult(urlparsed.scheme, urlparsed.netloc, urlparsed.path, urlparsed.params, urllib.parse.quote(urlparsed.query), urlparsed.fragment)
urlopen_url = urllib.parse.urlunparse(urlparsed_unicode_sanitized_query)

but this also fails if the query string is already url-encoded :(