Skip to content

Revert the output pipe in the DuplexStreamPipeAdapter #11601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 27, 2019

Conversation

davidfowl
Copy link
Member

  • Don't complete the connection pipe in Http2FrameWriter
  • Replace the output pipe only
  • Updated tests

Fixes #11560

- This leads to trunated data in some cases. Instead just yield the middleware so we can be sure no more user code is running (Http1OutputProducer does this as well). There are still cases where a misbeaving application that doesn't properly await writes gets cut off but that will be fixed in the SteamPipeWriter itself.
- Updated tests
}
catch (Exception ex)
{
Log?.LogError(0, ex, $"{GetType().Name}.{nameof(WriteOutputAsync)}");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that both LibuvConnection and SocketConnection will not complete their PipeReaders with an error unless it was completely unexpected, it's tempting to make this a critical log. That's unless we're worried about something like SslStream throwing an Exception in the middle of WriteAsync despite there being no transport issue.

Maybe that is something we should be worried about... I'm just extra worried about swallowing errors now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to leave it at error for now. I do agree that swallowing errors is a concern, but what else should we do. Debug.Assert? :trollface:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David already passed in a logger here. Critical logs are the new, much-improved Debug.Assert. Any critical log should cause the majority of our tests to fail.

readerScheduler: PipeScheduler.Inline,
writerScheduler: PipeScheduler.Inline,
pauseWriterThreshold: 1,
resumeWriterThreshold: 1,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably use the defaults here like we did before to reduce thrashing:

https://github.com/aspnet/AspNetCore/blob/release/3.0-preview6/src/Servers/Kestrel/Core/src/Internal/HttpConnection.cs#L67-L76

        internal PipeOptions AdaptedOutputPipeOptions => new PipeOptions
        (
            pool: MemoryPool,
            readerScheduler: PipeScheduler.Inline,
            writerScheduler: PipeScheduler.Inline,
            pauseWriterThreshold: _context.ServiceContext.ServerOptions.Limits.MaxResponseBufferSize ?? 0,
            resumeWriterThreshold: _context.ServiceContext.ServerOptions.Limits.MaxResponseBufferSize ?? 0,
            useSynchronizationContext: false,
            minimumSegmentSize: MemoryPool.GetMinimumSegmentSize()
        );

Copy link
Member

@halter73 halter73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the pauseWriterThreshold and resumeWriterThreshold before merging though.

@@ -89,7 +89,6 @@ public void Complete()

_completed = true;
_connectionOutputFlowControl.Abort();
_outputWriter.Complete();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@halter73
Copy link
Member

Maybe trying out the lower thresholds are OK for a preview. It does seem a little unecsary to try out new options now if we're removing it completely in the next preview though.

@davidfowl
Copy link
Member Author

davidfowl commented Jun 26, 2019

Maybe trying out the lower thresholds are OK for a preview. It does seem a little unecsary to try out new options now if we're removing it completely in the next preview though.

I'd rather leave it as it so its as passthrough as possible. Also the fact that it is temporary and the fact that those options aren't readily available (they need to be flowed from the UseHttps call) is just more churn. If you do feel strongly about changing those limits back to the Kestrel limits, I'll do it.

@halter73
Copy link
Member

halter73 commented Jun 26, 2019

If you do feel strongly about changing those limits back to the Kestrel limits, I'll do it.

I'm fine with the way it is since it's temporary. Otherwise, I'd at least ask for benchmarks.

@davidfowl
Copy link
Member Author

I'm fine with the way it is since it's temporary. Otherwise, I'd at least ask for benchmarks.

Maybe it's reasonable to leave the defaults then.

@davidfowl
Copy link
Member Author

I'm fine with the way it is since it's temporary. Otherwise, I'd at least ask for benchmarks.

I just realized though, you'll very likely never hit those limits since we always consume the entire buffer and the scheduling is inline. It means call to write will yield the await, write to the Stream then call Advance). It'll only matter when we're experiencing back pressure from the transport itself (which is the effect we want, complete passthrough).

@davidfowl davidfowl merged commit 6de357e into master Jun 27, 2019
@ghost ghost deleted the davidfowl/back-to-pipes branch June 27, 2019 05:06
@halter73
Copy link
Member

halter73 commented Jun 27, 2019

I just realized though, you'll very likely never hit those limits since we always consume the entire buffer and the scheduling is inline. It means call to write will yield the await, write to the Stream then call Advance). It'll only matter when we're experiencing back pressure from the transport itself (which is the effect we want, complete passthrough).

That has always been my assumption. I talked to @jkotalik about exactly this earlier today. That's why I'm not too worried. I like validating my assumptions though.

halter73 added a commit that referenced this pull request Jul 11, 2019
This reverts commit 6de357e.

# Conflicts:
#	src/Servers/Kestrel/Core/src/Internal/Http2/Http2FrameWriter.cs
#	src/Servers/Kestrel/Core/src/Middleware/Internal/DuplexPipeStreamAdapter.cs
halter73 added a commit that referenced this pull request Jul 12, 2019
This reverts commit 6de357e.

# Conflicts:
#	src/Servers/Kestrel/Core/src/Internal/Http2/Http2FrameWriter.cs
#	src/Servers/Kestrel/Core/src/Middleware/Internal/DuplexPipeStreamAdapter.cs
halter73 added a commit that referenced this pull request Jul 13, 2019
This reverts commit 6de357e.

# Conflicts:
#	src/Servers/Kestrel/Core/src/Internal/Http2/Http2FrameWriter.cs
#	src/Servers/Kestrel/Core/src/Middleware/Internal/DuplexPipeStreamAdapter.cs
halter73 added a commit that referenced this pull request Jul 16, 2019
This reverts commit 6de357e.

# Conflicts:
#	src/Servers/Kestrel/Core/src/Internal/Http2/Http2FrameWriter.cs
#	src/Servers/Kestrel/Core/src/Middleware/Internal/DuplexPipeStreamAdapter.cs
halter73 added a commit that referenced this pull request Jul 16, 2019
This reverts commit 6de357e.

# Conflicts:
#	src/Servers/Kestrel/Core/src/Internal/Http2/Http2FrameWriter.cs
#	src/Servers/Kestrel/Core/src/Middleware/Internal/DuplexPipeStreamAdapter.cs
@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Network errors when refreshing an app repeatedly
5 participants