Usage of HTTPResponse.url

[fbidu]

BPO	42062
Nosy	@vadmium, @fbidu
PRs	bpo-42062: Set HTTPResponse.url at init #22738

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-17.14:13:56.336>
labels = ['library']
title = 'Usage of HTTPResponse.url'
updated_at = <Date 2020-10-18.01:59:43.626>
user = 'https://github.com/fbidu'

bugs.python.org fields:

activity = <Date 2020-10-18.01:59:43.626>
actor = 'martin.panter'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-10-17.14:13:56.336>
creator = 'fbidu'
dependencies = []
files = []
hgrepos = []
issue_num = 42062
keywords = ['patch']
message_count = 2.0
messages = ['378814', '378846']
nosy_count = 2.0
nosy_names = ['martin.panter', 'fbidu']
pr_nums = ['22738']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue42062'
versions = []

[fbidu]

Hello all,

While testing some static analysis tools on HTTP/client.py, Pylint pointed
me to HTTPResponse.geturl() method with a "no-member" error for the url
attribute. I tried invoking the geturl method and reading the
HTTPResponse.url attribute using a sample code from the official docs:

import http.client
conn = http.client.HTTPSConnection("www.python.org")
conn.request("GET", "/")
r1 = conn.getresponse()
print(r1.status, r1.reason)

r1.geturl()
r1.url

import http.client
conn = http.client.HTTPSConnection("www.python.org")
conn.request("GET", "/")
r1 = conn.getresponse()
data1 = r1.read()

conn.request("GET", "/")
r1 = conn.getresponse()
while chunk := r1.read(200):
    print(repr(chunk))
r1.geturl()
r1.url

Both of those examples will raise an AttributeError: 'HTTPResponse' object has no attribute 'url'.

I tried searching through this module's history from when this line originally appeared,
6c5e28c but
I wasn't able to find this attribute being set internally by the class, even
though there is an url attribute at init.

So, I wonder if this attribute was intended to be set externally as in r1.url = 'something'
or if it is just a bug

[vadmium]

There is a comment in the HTTPResponse class regarding these methods:

# For compatibility with old-style urllib responses.

They were there for the "urlopen" API in "urllib.request", not for the "http.client" module on its own. I expect the "url" attribute is set by the "urlopen" code.

However more recently (bpo-12707) the "url" attribute and "geturl" method were documented in the HTTPResponse documentation, which is awkward.

[dimo414]

This seems to still be broken in 3.12:

$ docker run --rm python:3.12 python -c \
  'import http.client as httplib; conn = httplib.HTTPConnection("google.com"); conn.request("GET", "/"); print(conn.getresponse().geturl())'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.12/http/client.py", line 790, in geturl
    return self.url
           ^^^^^^^^
AttributeError: 'HTTPResponse' object has no attribute 'url'

HTTPResponse.url doesn't exist, and HTTPResponse.geturl() is broken. I tried 3.12, 3.11, 3.10, 3.9, and 3.8 (prior to #11447), it doesn't look like this has ever worked.

FYI @merwok and @matrixise who approved #11447.

[merwok]

Tangent: I now wish the original PR had also updated the docstrings on the deprecated methods (it only added a code comment).

[vadmium]

Another problem with this documentation is it suggests HTTPResponse.headers is an EmailMessage instance, while HTTPResponse.msg remains only a (subclassed) instance of email.message.Message. And addinfourl.info and addinfourl.headers are now supposed to return an EmailMessage, although .info was previously (indirectly) documented to return an email.message.Message.

I tested Python 3.10.2 and it seems HTTPResponse.headers has not changed because the object didn’t have one of the methods unique to EmailMessage:

>>> u = urlopen('http://google.com')
>>> u.headers.is_attachment
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'HTTPMessage' object has no attribute 'is_attachment'

[futurend]

Still present on Python 3.13.3 in 26 may 2025