Skip to content

Add QUIC_PERF_COUNTER_CONN_LOAD_REJECT counter. #4120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 14, 2024

Conversation

rzikm
Copy link
Member

@rzikm rzikm commented Feb 8, 2024

Description

This PR adds a counter which reports the number of connections which were automatically rejected on MsQuic layer because the the worker is already overloaded and is barely able to service current connections.

Testing

Tested by running .NET System.Net.Quic tests suite in parallel which exhibited this problem in the past.

Documentation

Documentation was added.

@rzikm rzikm requested a review from a team as a code owner February 8, 2024 13:41
@nibanks
Copy link
Member

nibanks commented Feb 8, 2024

I'm not opposed to this, but I'm curious why the Listener statistics isn't enough, since this is all in-proc for .NET.

@nibanks
Copy link
Member

nibanks commented Feb 8, 2024

Looks like we're getting a compile error:

D:\a\msquic\msquic\src\core\library.c(268,1): warning C4310: cast truncates constant value [D:\a\msquic\msquic\src\core\core.kernel.vcxproj]

This is coming from:

    QuicTraceEvent(
        PerfCountersRundown,
        "[ lib] Perf counters Rundown, Counters=%!CID!",
        CASTED_CLOG_BYTEARRAY(sizeof(PerfCounterSamples), PerfCounterSamples));

Which uses:

#define CLOG_BYTEARRAY(Len, Data) (uint8_t)(Len), (uint8_t*)(Data)
#define CASTED_CLOG_BYTEARRAY(Len, Data) CLOG_BYTEARRAY((unsigned char)(Len), (const unsigned char*)(Data))

The issue is that you just pushed past 256 bytes. I need to look at what needs to be changed to address this.

@nibanks
Copy link
Member

nibanks commented Feb 8, 2024

Ok, luckily the ETW event actually takes a 16-bit in, not 8-bit, so it's just a few macros we need to update I believe. Let's define

#define CLOG_BYTEARRAY16(Len, Data) (uint16_t)(Len), (uint8_t*)(Data)
#define CASTED_CLOG_BYTEARRAY16(Len, Data) CLOG_BYTEARRAY16((unsigned short)(Len), (const unsigned char*)(Data))

And use that in the two places you're getting errors.

Note - You will also need to define the version for all the other platforms. I think you can just copy/paste and make a similar update.

@rzikm
Copy link
Member Author

rzikm commented Feb 8, 2024

I'm not opposed to this, but I'm curious why the Listener statistics isn't enough, since this is all in-proc for .NET.

I am working on introducing some global diagnostic counters which can be observed using external tools (e.g. dotnet counters) and these diagnostics are process-wide. It is difficult to aggregate statistics from instance objects. We would probably have to maintain a list of all listeners and periodically pull and aggreagate them. And then listener statistics don't distinguish between load-rejected and app-rejected so we would have to add more tracking on our layer.

That aside, global perf counter for load-rejected connections seemed to me a good one to have in MsQuic anyway so I figured it would be better overall to add it here.

@nibanks
Copy link
Member

nibanks commented Feb 9, 2024

Looks like you need to update the CLOG files too now.

Copy link

codecov bot commented Feb 12, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (86e08e5) 87.38% compared to head (992d553) 64.97%.
Report is 27 commits behind head on main.

❗ Current head 992d553 differs from pull request most recent head 884b6d5. Consider uploading reports for the commit 884b6d5 to get more accurate results

Files Patch % Lines
src/core/listener.c 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #4120       +/-   ##
===========================================
- Coverage   87.38%   64.97%   -22.41%     
===========================================
  Files          56       56               
  Lines       16984    15187     -1797     
===========================================
- Hits        14841     9868     -4973     
- Misses       2143     5319     +3176     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rzikm rzikm requested a review from nibanks February 14, 2024 17:59
@nibanks nibanks merged commit b513e4b into microsoft:main Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants