Add URL with fragment to ClientResponse #2925

hdk5 · 2018-04-10T09:29:31Z

What do these changes do?

ClientResponse and RequestInfo now have real_url property, which is request url without fragment part being stripped

Checklist

I think the code is well written
Unit tests for the changes exist
Documentation reflects the changes
If you provide code modification, please add yourself to CONTRIBUTORS.txt
Add a new news fragment into the CHANGES folder

hdk5 · 2018-04-10T09:32:23Z

I had a trouble retrieving URL fragment from response URL. I am aware that request should be sent without URL fragment, but I don't think that URL from response should be modified.

codecov-io · 2018-04-10T10:20:03Z

Codecov Report

Merging #2925 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2925      +/-   ##
==========================================
+ Coverage   97.99%   97.99%   +<.01%     
==========================================
  Files          40       40              
  Lines        7514     7520       +6     
  Branches     1318     1318              
==========================================
+ Hits         7363     7369       +6     
  Misses         48       48              
  Partials      103      103

Impacted Files	Coverage Δ
aiohttp/client_reqrep.py	`97.27% <100%> (+0.02%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 544716c...aac2dc4. Read the comment docs.

asvetlov · 2018-04-10T10:41:17Z

HTTP response has no URL (unless it is not 3xx redirection).
ClientResponse.url is an URL of request.
I think that keeping a fragment part in response:

Makes inconsistency mess.
Breaks existing API

hdk5 · 2018-04-10T16:39:54Z

unless it is not 3xx redirection

This is what i meant. I need to know full URL I am getting redirected to because this is how I get application token.

Here is a little example: https://dry-meadow-80563.herokuapp.com/tokenask

It redirects to /tokengive#token=hereisyourtoken, and now it is not possible to get #token=hereisyourtoken part from response object.

hdk5 · 2018-04-10T16:40:35Z

aiohttp/client_reqrep.py

@@ -191,8 +191,8 @@ def __init__(self, method, url, *,
            url2 = url.with_query(params)
            q.extend(url2.query)
            url = url.with_query(q)
-        self.url = url.with_fragment(None)
        self.original_url = url


Also, this code before my commit looks strange to me:

self.url = url.with_fragment(None) self.original_url = url

Both url and original_url are not modified after (at least by the object itself), so they are always the same.

And then original_url is passed as url for ClientResponse.

So, wasn't it actually intended to keep URL fragment in response object?

asvetlov · 2018-04-11T08:15:40Z

Let's consider ClientRequest as private API. I don't care too much about its attributes.
The library needs a public client request object but it is another story.

The public API is ClientResponse and resp.history for looking on redirection URLs.
The history is a list of RequestInfo data structures.
I propose adding a new attribute called real_url (unstripped_url or whatever) to save URL with a fragment.

By this, we will keep the behavior of existing code and provide a way for getting original unmodified URLs.

Opinions?

hdk5 · 2018-04-11T08:31:31Z

I propose adding a new attribute called real_url

This might be a solution.

But can you please explain my previous comment on code and this commit?

asvetlov · 2018-04-11T08:39:16Z

Looks like @fafhrd91 wanted to implement exactly your initial intention, later we broke the behavior again.
I see two reasons for it:

Logical complexity: there is no obvious answer to the question: what is response's URL. I have a feeling that we need two properties.
The mentioned commit has no tests/documentation update, that's why we did break it insensibly.

hdk5 · 2018-04-11T09:08:55Z

Should I just put real url in real url attribute then, like you suggested, and update docs?

asvetlov · 2018-04-11T09:40:17Z

I think we need both ClientResponse.real_url and RequestInfo.real_url.

Documentation and tests are mandatory.

asvetlov · 2018-04-17T10:39:51Z

docs/client_reference.rst

+   .. attribute:: real_url
+
+      Unmodified URL of request (:class:`~yarl.URL`).
+


Please add .. versionadded:: 3.2 tag

asvetlov · 2018-04-17T10:40:07Z

docs/client_reference.rst

+   .. attribute:: real_url
+
+      Requested *url* with URL fragment unstripped, :class:`yarl.URL` instance.
+


.. versionadded:: 3.2

asvetlov · 2018-04-17T19:57:59Z

Thanks!

* Fixed ClientResponse URL fragment * Added news entry * ClientResponse real_url property + doc + test * Fixed flake8 and doc-spelling errors * Updated docs

lock · 2019-10-28T06:04:00Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

hdk5 commented Apr 10, 2018

View reviewed changes

hdk5 changed the title ~~Fixed ClientResponse URL fragment~~ Add URL with fragment to ClientResponse Apr 12, 2018

asvetlov requested changes Apr 17, 2018

View reviewed changes

hdk5 added 5 commits April 17, 2018 20:57

Fixed ClientResponse URL fragment

3fd0afb

Added news entry

45622df

ClientResponse real_url property + doc + test

2f819d9

Fixed flake8 and doc-spelling errors

f005304

Updated docs

aac2dc4

asvetlov approved these changes Apr 17, 2018

View reviewed changes

asvetlov merged commit cb17696 into aio-libs:master Apr 17, 2018

lock bot added the outdated label Oct 28, 2019

lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add URL with fragment to ClientResponse #2925

Add URL with fragment to ClientResponse #2925

hdk5 commented Apr 10, 2018 •

edited

Loading

hdk5 commented Apr 10, 2018

codecov-io commented Apr 10, 2018 •

edited

Loading

asvetlov commented Apr 10, 2018

hdk5 commented Apr 10, 2018

hdk5 Apr 10, 2018

asvetlov commented Apr 11, 2018

hdk5 commented Apr 11, 2018

asvetlov commented Apr 11, 2018

hdk5 commented Apr 11, 2018

asvetlov commented Apr 11, 2018

asvetlov Apr 17, 2018

asvetlov Apr 17, 2018

asvetlov commented Apr 17, 2018

lock bot commented Oct 28, 2019

		.. attribute:: real_url

		Unmodified URL of request (:class:`~yarl.URL`).

		.. attribute:: real_url

		Requested url with URL fragment unstripped, :class:`yarl.URL` instance.

Add URL with fragment to ClientResponse #2925

Add URL with fragment to ClientResponse #2925

Conversation

hdk5 commented Apr 10, 2018 • edited Loading

What do these changes do?

Checklist

hdk5 commented Apr 10, 2018

codecov-io commented Apr 10, 2018 • edited Loading

Codecov Report

asvetlov commented Apr 10, 2018

hdk5 commented Apr 10, 2018

hdk5 Apr 10, 2018

Choose a reason for hiding this comment

asvetlov commented Apr 11, 2018

hdk5 commented Apr 11, 2018

asvetlov commented Apr 11, 2018

hdk5 commented Apr 11, 2018

asvetlov commented Apr 11, 2018

asvetlov Apr 17, 2018

Choose a reason for hiding this comment

asvetlov Apr 17, 2018

Choose a reason for hiding this comment

asvetlov commented Apr 17, 2018

lock bot commented Oct 28, 2019

hdk5 commented Apr 10, 2018 •

edited

Loading

codecov-io commented Apr 10, 2018 •

edited

Loading