HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) #5461

saxenapranav · 2023-03-08T05:35:10Z

PR to backport merged pr #5299 in branch 3.3.
JIRA: https://issues.apache.org/jira/browse/HADOOP-18606

Commit of the merge in apache/trunk: 358bf80

it had only one conflict on cherry-pick:

<<<<<<< HEAD
      LOG.warn("Unknown host name: %s. Retrying to resolve the host name...",
=======
      failureReason = RetryReason.getAbbreviation(ex, null, null);
      LOG.warn("Unknown host name: {}. Retrying to resolve the host name...",
>>>>>>> 358bf80c945... HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299)
          hostname);

the conflict is resolved at:

hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java

Lines 317 to 321 in 08219d2

    
           hostname = httpOperation.getHost(); 
        
           failureReason = RetryReason.getAbbreviation(ex, null, null); 
        
           LOG.warn("Unknown host name: {}. Retrying to resolve the host name...", 
        
               hostname); 
        
           if (!client.getRetryPolicy().shouldRetry(retryCount, -1)) {

.

b75ced1 has been picked in the
saxenapranav:HADOOP-18606-branch-3.3

:::: AGGREGATED TEST RESULT ::::

HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:216 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 136, Failures: 1, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:84 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:143 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 568, Failures: 0, Errors: 3, Skipped: 99
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAbfsTerasort.test_120_terasort:262->executeStage:206 » IO The ownership o...
[INFO]
[ERROR] Tests run: 336, Failures: 0, Errors: 1, Skipped: 55

HNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:181->fuzzyValidate:64 The actual value 9 is not within the expected range: [5.60, 8.40].
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:216 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 136, Failures: 2, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=54).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 568, Failures: 1, Errors: 1, Skipped: 54
[INFO] Results:
[INFO]
[WARNING] Tests run: 336, Failures: 0, Errors: 0, Skipped: 41

NonHNS-SharedKey

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[ERROR] TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:181->fuzzyValidate:64 The actual value 10 is not within the expected range: [5.60, 8.40].
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:216 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 136, Failures: 2, Errors: 1, Skipped: 2
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:344 » TestTimedOut test timed o...
[INFO]
[ERROR] Tests run: 568, Failures: 0, Errors: 1, Skipped: 277
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:211->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89 teragen(1000, abfs://testcontainer@pranavsaxenanonhns.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed expected:<0> but was:<1>
[ERROR] Errors:
[ERROR] ITestAbfsJobThroughManifestCommitter.test_0420_validateJob » OutputValidation ...
[ERROR] ITestAbfsManifestCommitProtocol.testCommitLifecycle » OutputValidation abfs:/... [ERROR] ITestAbfsManifestCommitProtocol.testCommitterWithDuplicatedCommit » OutputValidation [ERROR] ITestAbfsManifestCommitProtocol.testConcurrentCommitTaskWithSubDir » OutputValidation [ERROR] ITestAbfsManifestCommitProtocol.testMapFileOutputCommitter » OutputValidation ... [ERROR] ITestAbfsManifestCommitProtocol.testOutputFormatIntegration » OutputValidation [ERROR] ITestAbfsManifestCommitProtocol.testParallelJobsToAdjacentPaths » OutputValidation [ERROR] ITestAbfsManifestCommitProtocol.testTwoTaskAttemptsCommit » OutputValidation ...
[INFO]
[ERROR] Tests run: 336, Failures: 1, Errors: 8, Skipped: 46

AppendBlob-HNS-OAuth

[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
[ERROR] Errors:
[ERROR] TestExponentialRetryPolicy.testOperationOnAccountIdle:216 » AccessDenied Opera...
[INFO]
[ERROR] Tests run: 136, Failures: 1, Errors: 1, Skipped: 1
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestAzureBlobFileSystemRandomRead.testSkipBounds:218->Assert.assertTrue:42->Assert.fail:89 There should not be any network I/O (elapsedTimeMs=224).
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemLease.testAcquireRetry:329 » TestTimedOut test timed o...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataContributor:84 » AccessDenied Operat...
[ERROR] ITestAzureBlobFileSystemOauth.testBlobDataReader:143 » AccessDenied Operation ...
[INFO]
[ERROR] Tests run: 568, Failures: 1, Errors: 3, Skipped: 99
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] ITestAbfsTerasort.test_120_terasort:262->executeStage:206 » IO The ownership o...
[INFO]
[ERROR] Tests run: 336, Failures: 0, Errors: 1, Skipped: 55

Time taken: 42 mins 17 secs.

hadoop-yetus · 2023-03-08T07:17:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	35m 55s		branch-3.3 passed
+1 💚	compile	0m 37s		branch-3.3 passed
+1 💚	checkstyle	0m 35s		branch-3.3 passed
+1 💚	mvnsite	0m 43s		branch-3.3 passed
+1 💚	javadoc	0m 39s		branch-3.3 passed
+1 💚	spotbugs	1m 17s		branch-3.3 passed
+1 💚	shadedclient	24m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	compile	0m 28s		the patch passed
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 32s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	spotbugs	1m 2s		the patch passed
+1 💚	shadedclient	23m 31s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 1s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		101m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/1/artifact/out/Dockerfile
GITHUB PR	#5461
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 056ff9ce20d5 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `08219d2`
Default Java	Private Build-1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/1/testReport/
Max. process+thread count	560 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

saxenapranav · 2023-03-08T08:45:34Z

@steveloughran, as discussed on #5299. I have backported the change in branch-3.3 Requesting you to kindly review it. Thank you so much.

steveloughran · 2023-03-08T15:19:55Z

ok. regarding that conflict, looks like it is because HADOOP-17836/ #3281 never got backported. I think I would like that in...let me pull it into branch-3.3 and then you can try to cherrypick again

hadoop-yetus · 2023-03-09T05:47:52Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	36m 20s		branch-3.3 passed
+1 💚	compile	0m 37s		branch-3.3 passed
+1 💚	checkstyle	0m 35s		branch-3.3 passed
+1 💚	mvnsite	0m 43s		branch-3.3 passed
+1 💚	javadoc	0m 39s		branch-3.3 passed
+1 💚	spotbugs	1m 17s		branch-3.3 passed
+1 💚	shadedclient	24m 16s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	24m 38s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 37s		the patch passed
+1 💚	compile	0m 28s		the patch passed
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 19s		the patch passed
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	spotbugs	1m 2s		the patch passed
+1 💚	shadedclient	23m 23s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 4s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		95m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/2/artifact/out/Dockerfile
GITHUB PR	#5461
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux efc01f079e87 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `829a9bd`
Default Java	Private Build-1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/2/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5461/2/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

saxenapranav · 2023-03-09T05:52:30Z

ok. regarding that conflict, looks like it is because HADOOP-17836/ #3281 never got backported. I think I would like that in...let me pull it into branch-3.3 and then you can try to cherrypick again

Thank you so much @steveloughran for cherry-picking the mentioned PR into branch-3.3. I have backmerged branch-3.3 into this PR. Requesting you to kindly review the PR please.

Thanks.

saxenapranav · 2023-03-20T13:37:17Z

Hi @steveloughran , thank you for helping in resolving conflict. Requesting you to kindly review the PR please. This PR is pre-requisite for merging changes from #5437 in branch-3.3

Regards.

saxenapranav · 2023-03-27T12:51:18Z

Hi @steveloughran , requesting you to kindly review the PR please. This is pre-requisite for merging changes from following PRs in 3.3:

HADOOP-18647. x-ms-client-request-id to have some way that identifies retry of an API. #5437
HADOOP-18146: ABFS: Added changes for expect hundred continue header #4039

Thank you.

steveloughran · 2023-03-28T11:03:01Z

merged.

in future can you give the pr the same title as the patch committed to trunk (including the # PR value) and the commit that and the body of the commit. because I end up having to chase that down on merge, and it both wastes time and increases the risk of errors.
thanks

cherrypicking apache/hadoop commit 358bf80: merged pr: apache#5299

08219d2

saxenapranav marked this pull request as ready for review March 8, 2023 05:35

backmerged branch-3.3:: apache@b75ced1

829a9bd

saxenapranav changed the title ~~Backport Merged pr https://github.com/apache/hadoop/pull/5299 in branch-3.3~~ HADOOP-18606. Backport Merged pr https://github.com/apache/hadoop/pull/5299 in branch-3.3 Mar 20, 2023

saxenapranav mentioned this pull request Mar 21, 2023

HADOOP-18647. x-ms-client-request-id to have some way that identifies retry of an API. (#5437) #5498

Merged

steveloughran changed the title ~~HADOOP-18606. Backport Merged pr https://github.com/apache/hadoop/pull/5299 in branch-3.3~~ HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) Mar 28, 2023

steveloughran approved these changes Mar 28, 2023

View reviewed changes

steveloughran merged commit 2b156c2 into apache:branch-3.3 Mar 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) #5461

HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) #5461

Uh oh!

saxenapranav commented Mar 8, 2023 •

edited

Loading

Uh oh!

hadoop-yetus commented Mar 8, 2023

Uh oh!

saxenapranav commented Mar 8, 2023

Uh oh!

steveloughran commented Mar 8, 2023

Uh oh!

hadoop-yetus commented Mar 9, 2023

Uh oh!

saxenapranav commented Mar 9, 2023

Uh oh!

saxenapranav commented Mar 20, 2023

Uh oh!

saxenapranav commented Mar 27, 2023

Uh oh!

steveloughran commented Mar 28, 2023

Uh oh!

Uh oh!

	hostname = httpOperation.getHost();
	failureReason = RetryReason.getAbbreviation(ex, null, null);
	LOG.warn("Unknown host name: {}. Retrying to resolve the host name...",
	hostname);
	if (!client.getRetryPolicy().shouldRetry(retryCount, -1)) {

HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) #5461

HADOOP-18606. ABFS: Add reason in x-ms-client-request-id on a retried API call. (#5299) #5461

Uh oh!

Conversation

saxenapranav commented Mar 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

HNS-OAuth

HNS-SharedKey

NonHNS-SharedKey

AppendBlob-HNS-OAuth

Uh oh!

hadoop-yetus commented Mar 8, 2023

Uh oh!

saxenapranav commented Mar 8, 2023

Uh oh!

steveloughran commented Mar 8, 2023

Uh oh!

hadoop-yetus commented Mar 9, 2023

Uh oh!

saxenapranav commented Mar 9, 2023

Uh oh!

saxenapranav commented Mar 20, 2023

Uh oh!

saxenapranav commented Mar 27, 2023

Uh oh!

steveloughran commented Mar 28, 2023

Uh oh!

Uh oh!

saxenapranav commented Mar 8, 2023 •

edited

Loading