Upgrading `http.client` to use http email policy for parsing

[michaelfm1211]

To parse HTTP headers, the http.client module uses the email.parser.Parser class, as headers are in the same format. The relevant function for this is _parse_header_lines. Because http.server uses http.client to parse its headers, this function is also responsible for parsing headers on the server side.

However, _parse_header_lines doesn't pass an email policy to email.parser.Parser. The documentation mentions that a policy should always be provided, and that the default policy will change in the future, so it's probably a good idea to change the code to use a specific policy.

The current default policy for email.parser.Parser is email.policy.compat32, which is a bit outdated but provides complete backwards compatibility. Because the documentation says that this won't be the case forever, I suggest that we change _parse_header_lines to use email.policy.default instead. While this is a breaking change, it would greatly extend support for the many extra RFCs that email.policy.default already supports.

One of these RFCs in particular that this change would start supporting is RFC 2047, which includes receiving encoded-word header values and would resolve part of issue #105530.

Note: this is also on discuss.python.org.

Linked PRs

• gh-105622: Parse HTTP headers with default email policy #105918

[vstinner]

I close the issue since the PR has been closed.

[bitdancer]

I'm not sure why you closed this issue? This is something that needs to be done, sooner or later. Although the policy that should be used is the http policy, that's what it's designed for.

[vstinner]

I closed the PR and the issue since the issue reporter, who is also the PR author, was idle for 1 year, didn't sign the CLA, and didn't address my review.

[bitdancer]

Fair enough. However, this is something that should be done, so I'd like to keep the issue open. It will be a while before I could do anything about it, unfortunately.