Skip to content

Commit

Permalink
Parse the last CRLF of chunked response correctly (aio-libs#4630)
Browse files Browse the repository at this point in the history
If the last CRLF or only the LF are received via separate TCP segment,
HTTPPayloadParser misjudges that trailers should come after 0\r\n in the
chunked response body.

In this case, HttpPayloadParser starts waiting for trailers, but the only
remaining data to be received is CRLF. Thus, HttpPayloadParser waits trailers
indefinitely and this incurs TimeoutError in user code.

However, if the connection is keep alive disabled, this problem is not
reproduced because the server shutdown the connection explicitly after sending
all data. If the connection is closed .feed_eof is called and it helps
HttpPayloadParser finish its waiting.
  • Loading branch information
rhdxmr committed Jun 10, 2020
1 parent 385b03e commit 693f2ce
Show file tree
Hide file tree
Showing 3 changed files with 16 additions and 3 deletions.
1 change: 1 addition & 0 deletions CHANGES/4630.bugfix
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Handle the last CRLF correctly even if it is received via separate TCP segment.
1 change: 1 addition & 0 deletions CONTRIBUTORS.txt
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ Julia Tsemusheva
Julien Duponchelle
Jungkook Park
Junjie Tao
Junyeong Jeong
Justas Trimailovas
Justin Foo
Justin Turner Arthur
Expand Down
17 changes: 14 additions & 3 deletions aiohttp/http_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -668,12 +668,23 @@ def feed_data(self,
# we should get another \r\n otherwise
# trailers needs to be skiped until \r\n\r\n
if self._chunk == ChunkState.PARSE_MAYBE_TRAILERS:
if chunk[:2] == SEP:
head = chunk[:2]
if head == SEP:
# end of stream
self.payload.feed_eof()
return True, chunk[2:]
else:
self._chunk = ChunkState.PARSE_TRAILERS
# Both CR and LF, or only LF may not be received yet. It is
# expected that CRLF or LF will be shown at the very first
# byte next time, otherwise trailers should come. The last
# CRLF which marks the end of response might not be
# contained in the same TCP segment which delivered the
# size indicator.
if not head:
return False, b''
if head == SEP[:1]:
self._chunk_tail = head
return False, b''
self._chunk = ChunkState.PARSE_TRAILERS

# read and discard trailer up to the CRLF terminator
if self._chunk == ChunkState.PARSE_TRAILERS:
Expand Down

0 comments on commit 693f2ce

Please sign in to comment.