HADOOP-18364. All method metrics related to the RPC protocol should be initialized. #4624
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zhangshuyan0
force-pushed
the
initMetric
branch
2 times, most recently
from
30c79ce to
7f8eef0
Compare
Hexiaoqiao
changed the title
Hexiaoqiao
reviewed
Jul 25, 2022
There was a problem hiding this comment.
@zhangshuyan0 Thanks for your report and contribution. It makes sense to me. Just find that workflow report one conflict which should be fixed first.
cc @sunchao @jojochuang would you mind have another check? Thanks.
goiri
changed the title
sunchao
reviewed
Jul 25, 2022
|
|
||
| List<Class<?>> protocols = new ArrayList<>(); | ||
| protocols.add(protocol); | ||
| if (protocol.getDeclaredMethods().length == 0) { |
There was a problem hiding this comment.
make add some comments for this since it's not very obvious why we do it.
sunchao
approved these changes
Jul 25, 2022
xkrogen
requested changes
Jul 25, 2022
There was a problem hiding this comment.
Interesting to hear that we're passing the XXXProtocolPB classes which don't declare any methods. I think when testing this I must have assumed we passed the XXXProtocol or XXXTranslatorPB classes, though it's been too long for me to remember.
Can you add testing for this? A simple test in TestMutableMetrics would be beneficial to validate the method-handling logic; we can augment TestProtocol there to have some inheritance to create a similar situation as real protocols.
If possible, I'd also love to see a higher-level test validating that it works on a real-world protocol, which should be broken by the current logic. Something like starting a NameNode, making one method call, and validating that metrics exist for all of the other methods in the protocol containing that method call.
.../hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableRatesWithAggregation.java
Outdated
Show resolved
Hide resolved
|
💔 -1 overall
This message was automatically generated. |
Hexiaoqiao
reviewed
Aug 1, 2022
| Configuration conf = new HdfsConfiguration(); | ||
| // setting heartbeat interval to 1 hour to prevent bpServiceActor sends | ||
| // heartbeat periodically to NN during running test case, and bpServiceActor | ||
| // only sends heartbeat once after startup | ||
| conf.setTimeDuration(DFS_HEARTBEAT_INTERVAL_KEY, 1, TimeUnit.HOURS); | ||
| MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).build(); | ||
| cluster.waitActive(); | ||
| final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer(); |
There was a problem hiding this comment.
@zhangshuyan0 Thanks for your update. This unit test seems failed based on the yetus reports. Another side, this segment code located at TestNameNodeMetrics should be more proper? Thanks again.
There was a problem hiding this comment.
Agreed that this isn't the right place. Actually it seems like enhancing TestRPC#testRpcMetrics() might be the best place? Or if we can't get it to work there, then I agree TestNameNodeMetrics seems better.
zhangshuyan0
force-pushed
the
initMetric
branch
from
f8d341e to
b79ea58
Compare
xkrogen
requested changes
Aug 2, 2022
...p-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/metrics/TestNameNodeMetrics.java
Show resolved
Hide resolved
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
@xkrogen Would you mind to take another review? I will to check in this PR after you confirm. Thanks. |
xkrogen
approved these changes
Aug 3, 2022
Hexiaoqiao
changed the title
|
Committed to trunk. Thanks @zhangshuyan0 for your works. Thanks @sunchao, @xkrogen for your nice reviews! |
HarshitGupta11
pushed a commit
to HarshitGupta11/hadoop
that referenced
this pull request
Nov 28, 2022
…e initialized. (apache#4624). Contributed by Shuyan Zhang. Reviewed-by: Erik Krogen <xkrogen@apache.org> Reviewed-by: Chao Sun <sunchao@apache.org> Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…initialized.
Description of PR
How was this patch tested?
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?