Replace DataQueue with StreamReader for request payload

[fafhrd91]

This is outcome of #81, Replace DataQueue with StreamReader for request payload

[fafhrd91]

added branch with experimental work https://github.com/KeepSafe/aiohttp/tree/streamreader-payload

@asvetlov what do you think?

[fafhrd91]

i'm going to merge this branch to master.

@asvetlov @popravich @kxepal objections?

[asvetlov]

Sorry, I still have no time for review.
Will do it up to end of Sunday.

[popravich]

looks good, no objections

[kxepal]

Hm. While looks good, it breaks some my code. Currently, if you call ClientResponse.content.read() it reads and emits one single chunk. This allowed to process chunked response chunk by chunk without worry about the details. With streamparser-payload changes it reads and returns all available chunks in one shot and doesn't tries to handle following ones on the next call. However, I didn't looks carefully yet for alternative solution on feature branch, may be I should rewrite some things in different way.

[fafhrd91]

accidentally removed comment :(

[kxepal]

Yea, figured that. Was easy to fix.

However, content.readany() isn't proper replacement since still it squashes multiple chunks into one big bytearray. Partially replacement for me was using content.readline() instead, but it makes handling EventSource a bit complex since it's all about new lines. I wonder, may be add content.readchunk() method specially for chunked transfer response? Will try to play around to make PR about if you don't mind.

[kxepal]

Was unable to make readchunk() method since it would require to break some things for almost no reason. However, I found that FlowControlDataQueue is perfectly fits the goal and the only what it need is to make it replaceable for FlowControlStreamReader. Check #100 for the related changes.

[asvetlov]

LGTM!

[fafhrd91]

merged

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.