Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#1686] feat(netty): Support pending tasks number metrics for Netty EventLoopGroup #1687

Merged
merged 5 commits into from
May 11, 2024

Conversation

rickyma
Copy link
Contributor

@rickyma rickyma commented May 9, 2024

What changes were proposed in this pull request?

Support pending tasks number metrics for Netty EventLoopGroup.

Why are the changes needed?

For #1686.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing tests.

@@ -89,6 +89,13 @@ public class ShuffleServerConf extends RssBaseConf {
.defaultValue(10 * 1000L)
.withDescription("Direct memory usage tracker interval to MetricSystem (ms)");

public static final ConfigOption<Long> SERVER_NETTY_PENDING_TASKS_NUM_TRACKER_INTERVAL =
ConfigOptions.key("rss.server.netty.pending.tasks.num.updateMetricsIntervalMs")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use a more brief name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe NETTY_PENDING_TASKS_NUM_TRACKER_INTERVAL?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I means rss.server.netty.pending.tasks.num.updateMetricsIntervalMs

Copy link
Contributor Author

@rickyma rickyma May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the naming of rss.server.netty.directMemoryTracker.memoryUsage.updateMetricsIntervalMs.

Maybe rss.server.netty.pending.tasks.num.polling.interval? If we continue to simplify the naming of this variable, I'm afraid it might become difficult to understand.

Copy link
Contributor

@jerqi jerqi May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about rss.server.netty.metrics.pendingTaskNumPollingIntervalMs? Only the last name is camel. The former name segments can use more common word.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Done.

Copy link

github-actions bot commented May 9, 2024

Test Results

 2 391 files  ± 0   2 391 suites  ±0   4h 41m 32s ⏱️ +36s
   926 tests + 1     925 ✅ + 1   1 💤 ±0  0 ❌ ±0 
10 726 runs  +14  10 712 ✅ +14  14 💤 ±0  0 ❌ ±0 

Results for commit cb85bc3. ± Comparison against base commit 30bf8dc.

♻️ This comment has been updated with latest results.

@rickyma rickyma requested a review from jerqi May 10, 2024 02:45
jerqi
jerqi previously approved these changes May 10, 2024
Copy link
Contributor

@jerqi jerqi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Do you think we should include this in the branch 0.9?

@rickyma
Copy link
Contributor Author

rickyma commented May 10, 2024

LGTM, Do you think we should include this in the branch 0.9?

Either is fine. Of course it will be nice to have this feature.

@codecov-commenter
Copy link

codecov-commenter commented May 10, 2024

Codecov Report

Attention: Patch coverage is 81.08108% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 54.57%. Comparing base (6f6d35a) to head (2ccf5b3).
Report is 9 commits behind head on master.

Files Patch % Lines
...rg/apache/uniffle/common/metrics/NettyMetrics.java 0.00% 6 Missing ⚠️
.../org/apache/uniffle/server/netty/StreamServer.java 96.15% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1687      +/-   ##
============================================
- Coverage     54.86%   54.57%   -0.29%     
- Complexity     2358     2750     +392     
============================================
  Files           368      411      +43     
  Lines         16379    21483    +5104     
  Branches       1504     2031     +527     
============================================
+ Hits           8986    11725    +2739     
- Misses         6862     9014    +2152     
- Partials        531      744     +213     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jerqi
Copy link
Contributor

jerqi commented May 10, 2024

Could you add the document for this config option?

@rickyma
Copy link
Contributor Author

rickyma commented May 10, 2024

Could you add the document for this config option?

We missed docs for rss.server.netty.directMemoryTracker.memoryUsage.initialFetchDelayMs and rss.server.netty.directMemoryTracker.memoryUsage.updateMetricsIntervalMs too. Is this doc for interval configurations necessary?

@jerqi
Copy link
Contributor

jerqi commented May 10, 2024

Could you add the document for this config option?

We missed docs for rss.server.netty.directMemoryTracker.memoryUsage.initialFetchDelayMs and rss.server.netty.directMemoryTracker.memoryUsage.updateMetricsIntervalMs too. Is this doc for interval configurations necessary?

Yes.

@rickyma
Copy link
Contributor Author

rickyma commented May 10, 2024

Added. But only for my new added config. The markdown table is reformatted.

@rickyma rickyma requested a review from jerqi May 10, 2024 08:32
@rickyma
Copy link
Contributor Author

rickyma commented May 10, 2024

If you want to add other configs in this PR. I'll be OK with that. Or use a new issue to track configs without docs.

@jerqi jerqi merged commit 7b63177 into apache:master May 11, 2024
41 checks passed
@jerqi
Copy link
Contributor

jerqi commented May 11, 2024

Merged to master & branch 0.9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants