Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-32251: Implement asyncio.BufferedProtocol. #4755

Merged
merged 14 commits into from
Jan 28, 2018
Merged

Conversation

1st1
Copy link
Member

@1st1 1st1 commented Dec 8, 2017

@asvetlov
Copy link
Contributor

asvetlov commented Dec 8, 2017

The idea is clean, I like it.
aiohttp can be adapted to buffered protocols usage easy. The same is true for other aio-libs projects like aioredis.


try:
buf = self._protocol.get_buffer()
with memoryview(buf) as mem:
Copy link
Member

@pitrou pitrou Dec 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't need a memoryview here, just pass buf to recv_into.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right

return

try:
buf = self._protocol.get_buffer()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be inside the try...except?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should. Any unhandled exception in get_buffer() means that the protocol is in an inconsistent state, and that it's safer to close the transport and shut everything down. Speaking of which, we should put buffer_updated and data_received in similar try..except blocks.

@1st1
Copy link
Member Author

1st1 commented Jan 28, 2018

@asvetlov Andrew, can you please take a look? If you are OK with this PR, I'll add docs & a functional test or two.

Copy link
Contributor

@asvetlov asvetlov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general but I have a couple minor questions.

Functional tests with reduced SO_RECVBUF size would be great!

'protocol': self._protocol,
})
self._force_close(exc)
try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like not related to PR subject but I love the change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the proactor code hasn't received any love recently, so there're many things we can improve here. I'm just fixing things I see slightly broken.

@@ -159,6 +168,11 @@ def pause_reading(self):
if self._closing or self._paused:
return
self._paused = True

if self._read_fut is not None and not self._read_fut.done():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proactor tricks again?
Not sure if cancelling is what we should do here.
self._read_fut should be set on next resume_reading call maybe? No sure though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think this is just a bug. The problem is that when you call pause_reading, the current code doesn't actually pause it. It just sets a _paused flag but leaves the socket in the reading mode. Do you agree, or am I missing something here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you are right

@@ -170,11 +184,25 @@ def resume_reading(self):
if self._loop.get_debug():
logger.debug("%r resumes reading", self)

def _loop_reading(self, fut=None):
def _loop_reading__on_eof(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is two underscore in name intentional?
We never use this code style in asyncio

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a bit more readable this way, since it's one "family" of functions (all three of them are combined in one "mega" function). I can replace with one underscore, but I think it's OK this way too. Up to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, agree

@1st1
Copy link
Member Author

1st1 commented Jan 28, 2018

Alright, I'll work on adding functional tests then.

@1st1
Copy link
Member Author

1st1 commented Jan 28, 2018

Just ran uvloop functional tests with updated asyncio: everything appears to be normal.

@1st1 1st1 merged commit 631fd38 into python:master Jan 28, 2018
@bedevere-bot
Copy link

@1st1: Please replace # with GH- in the commit message next time. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants