Upgrading `http.client` to use default email policy for parsing

[michaelfm1211]

To parse HTTP headers, the http.client module uses the email.parser.Parser class, as headers are in the same format. The relevant function for this is _parse_header_lines. Because http.server uses http.client to parse its headers, this function is also responsible for parsing headers on the server side.

However, _parse_header_lines doesn't pass an email policy to email.parser.Parser. The documentation mentions that a policy should always be provided, and that the default policy will change in the future, so it's probably a good idea to change the code to use a specific policy.

The current default policy for email.parser.Parser is email.policy.compat32, which is a bit outdated but provides complete backwards compatibility. Because the documentation says that this won't be the case forever, I suggest that we change _parse_header_lines to use email.policy.default instead. While this is a breaking change, it would greatly extend support for the many extra RFCs that email.policy.default already supports.

One of these RFCs in particular that this change would start supporting is RFC 2047, which includes receiving encoded-word header values and would resolve part of issue #105530.

Note: this is also on discuss.python.org.

Linked PRs

• gh-105622: Parse HTTP headers with default email policy #105918