gh-102327: Extend docs for "url" and "headers" parameters to HTTPConnection.request()

[davidfstr]

Added example on how to use the HTTPConnection object for making GET request.

Original issue: #102327

• Issue: Clarify that "url" parameter to HTTPConnection.request(method, url) is usually an absolute path #102327

[Mariatta]

Looks good overall, I think this provides clarity to the doc.

One comment, and I'm being nitpicky here, can we use a different URL in the example, instead of xkcd? Can we use python.org as example? Or other Python-related website.

[davidfstr]

Can we use python.org as example? Or other Python-related website.

Sure. I'll tentatively put in: https://docs.python.org/3/

That URL satisfies the following useful conditions:

• is owned by Python contributors
• uses "https" rather than "http", which promotes "https"
• uses a non-trivial path "/3/" (rather than just "/")

[merwok]

What do you think of moving the doc for url higher up (so that order of doc paragraphs matches order of parameters), and not using a note markup (which grabs attention and takes readers out of normal reading flow)?

The example could be kept at the end, after the documentation of each parameter.

[davidfstr]

@merwok , makes sense. I'll plan to integrate those revisions later today.

[davidfstr]

Revisions integrated. Here's the updated graphical diff of the proposed changes:

[merwok]

This looks pretty great now!

Thanks for the screenshots. Note that pull requests have automatic preview websites for docs, which are even better to see results easily (although sometimes the link is a bit hidden):
https://cpython-previews--102328.org.readthedocs.build/en/102328/library/http.client.html#httpconnection-objects

[merwok]

This line makes me wonder if PUT and PATCH are part of «most HTTP methods», so I have to follow the RFC link to see. It seems that yes, only OPTIONS * is given as a counter-example. I wonder if there is a way to rephrase that so that casual readers take away that url should nearly always be an absolute path, see link for exact rules.

What do other people think?

[davidfstr]

A more specific phrasing would be:

The provided *url* must be an absolute path
to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`,
unless connecting to an HTTP proxy server or
using the ``OPTIONS`` or ``CONNECT`` methods.

And further down:

A :rfc:`Host header <2616#section-14.23>` must be provided
to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`,
unless connecting to an HTTP proxy server or
using the ``OPTIONS`` or ``CONNECT`` methods.

A reader would still have to follow the link to determine the actual rules if they were talking to an HTTP proxy server (somewhat common?) or using OPTIONS/CONNECT (rare?).

Comments?

[merwok]

That’s very clear! The unless clause could even be in parentheses.

I never use proxies but I think there are two cases:

• explicit proxy, when you send GET https://authority/path on a connection opened to http://proxy
• transparent proxy, when you use an HTTP library that respects the http_proxy or https_proxy environment variables

I suppose the note here applies to the first kind only?

[davidfstr]

Feedback applied. I think this change is ready to merge!

[merwok]

Could you reply to the question about proxy? 🙂

[davidfstr]

RFC 2616 §1.3 defines a "proxy" as:

An intermediary program which acts as both a server and a client
for the purpose of making requests on behalf of other clients.
Requests are serviced internally or by passing them on, with
possible translation, to other servers. A proxy MUST implement
both the client and server requirements of this specification. A
"transparent proxy" is a proxy that does not modify the request or
response beyond what is required for proxy authentication and
identification. A "non-transparent proxy" is a proxy that modifies
the request or response in order to provide some added service to
the user agent, such as group annotation services, media type
transformation, protocol reduction, or anonymity filtering. Except
where either transparent or non-transparent behavior is explicitly
stated, the HTTP proxy requirements apply to both types of
proxies.

Therefore I speculate that the following requirement from §5.1.2 applies when a Python program attempts to connect to any kind of proxy:

The absoluteURI form is REQUIRED when the request is being made to a
proxy.

I never use proxies myself so I have no empirical experience one way or the other.

[merwok]

OK, let’s hope the people using proxies know how to handle them!

[merwok]

Note for the dev who will merge: please edit the commit message to avoid reusing the whole original post here with its questions and screenshot

[davidfstr]

What are the next steps to move this review forward?

• This review was previously approved by python maintainers @Mariatta and @AA-Turner , but that approval was reset when I cleaned up the PR title.
• This review was also approved by @merwok. (Thanks for the helpful revisions!)

(1) Would it be helpful if I squashed/rebased this commit to the tip of the main branch?

(2) Do I need to summon someone with merge permissions? If so, is there a list of such folks publicly listed?

[Mariatta]

I will re-review later today from computer. Thanks.

[merwok]

The approval was reset by me actually, not a title change!

Note that all PRs for CPython are squash merged, so rebases are not needed. (In fact the devguide recommends against them, as force pushes do weird things for reviewers: ghost notifications, comments losing context)

[Mariatta]

Thank you!

[miss-islington]

Thanks @davidfstr for the PR, and @Mariatta for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖