Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WebSockets telemetry #1237

Merged
merged 6 commits into from
Sep 30, 2021
Merged

Conversation

MihaZupan
Copy link
Member

@MihaZupan MihaZupan commented Sep 8, 2021

Fixes #1209

I followed the approach outlined in #1209 of wrapping the upgrade feature and MITM the upgrade transport stream.

Public API is just the UseWebSocketsTelemetry extension method.
I fed the data to an EventSource event, but it could also alternatively be something like IWebSocketsTelemetryFeature.

I didn't know where to put the code so I made a WebSocketsTelemetry folder in the main project (all types are internal though).

ToDo: I will add tests when we're happy with the overall approach

@MihaZupan MihaZupan added this to the YARP 1.0.0 milestone Sep 8, 2021
Copy link
Member

@Tratcher Tratcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the right structure. Testing is going to be interesting.

Work with the customer to decide the right field types for the telemetry output.

var clientCloseTime = _readParser.CloseTime;
var serverCloseTime = _writeParser.CloseTime;

if (clientCloseTime.HasValue && (!serverCloseTime.HasValue || serverCloseTime.Value > clientCloseTime.Value))
Copy link
Member

@Tratcher Tratcher Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The close is only graceful if both sides acknowledge it. Otherwise you can attribute the reason as a disconnect by the unacknowledged side.

Suggested change
if (clientCloseTime.HasValue && (!serverCloseTime.HasValue || serverCloseTime.Value > clientCloseTime.Value))
if (clientCloseTime.HasValue && serverCloseTime.HasValue && serverCloseTime.Value > clientCloseTime.Value)

https://datatracker.ietf.org/doc/html/rfc6455#section-7.1.2

Once an endpoint has both sent and
received a Close control frame, that endpoint SHOULD Close the
WebSocket Connection
as defined in

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So something like

if (clientSentClose && serverSentClose)
{
    return clientCloseTime < serverCloseTime ? ClientGraceful : ServerGraceful;
}
else
{
    Disconnect
}

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (clientSentClose && serverSentClose)
{
    return clientCloseTime < serverCloseTime ? ClientGraceful : ServerGraceful;
}
else if (clientSentClose)
{
    ServerDisconnect
}
else if (serverSentClose)
{
   ClientDisconnect
}
else // check forwarder errors

Copy link
Member Author

@MihaZupan MihaZupan Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the client sends a Close message and immediately closes the connection (without waiting for the server to respond), we would be reporting that as a ServerDisconnect (and vice-versa).

Would that be the desired behavior?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... how would you detect that? Would the read return 0? Writes might fail, depending on the timing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the logic so that it is now

if (clientSentClose && serverSentClose)
{
    return clientCloseTime < serverCloseTime ? ClientGraceful : ServerGraceful;
}

// One side sent a WebSocket close, but we never saw a response from the other side
// It is possible an error occurred, but we saw a graceful close first, so that is the intiator
if (clientSentClose)
{
    return ClientGraceful;
}
if (serverSentClose)
{
    return ServerGraceful;
}

return forwarderError ...

While "graceful" doesn't necessarily mean "both sides sent close", I believe reporting what likely initiated the close is more useful as the WebSocketCloseReason.
Either side may still perform protocol violations, but it's not our aim to catch that.

@@ -576,7 +576,7 @@ private static void RestoreUpgradeHeaders(HttpContext context, HttpResponseMessa
var (secondResult, secondException) = await secondTask;
if (secondResult != StreamCopyResult.Success)
{
error = ReportResult(context, requestFinishedFirst, secondResult, secondException!);
error = ReportResult(context, !requestFinishedFirst, secondResult, secondException!);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be a bug where if the request copy finishes first successfully, we will attribute potential response errors to the request side.

@MihaZupan MihaZupan modified the milestones: YARP 1.0.0, YARP RC1 Sep 21, 2021
@adityamandaleeka
Copy link
Member

@MihaZupan Are you waiting for anything from anyone on this PR?

Copy link
Member

@Tratcher Tratcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting some unit tests for the WebSocketsParser?

@MihaZupan
Copy link
Member Author

Added more tests

Copy link
Member

@Tratcher Tratcher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried running this in the sample with real websocket traffic?

@MihaZupan
Copy link
Member Author

Yes, I tried it with a simple demo app and the default Blazor template, and it's working as expected.

@MihaZupan MihaZupan merged commit c5f1607 into microsoft:main Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WebSockets insights
4 participants