HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi… #6343

ThinkerLei · 2023-12-11T13:53:43Z

When the Observer NameNode is turned on in the cluster, the Active NameNode will receive rollEditLog RPC requests from the Standby NameNode and Observer NameNode in a short time. Observer NameNode's rollEditLog request is a repetitive operation, so should we forbid Observer NameNode trigger active namenode roll editsLog

hadoop-yetus · 2023-12-11T18:24:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 52s		trunk passed
+1 💚	compile	0m 43s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 38s		trunk passed
+1 💚	mvnsite	0m 39s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 4s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 55s		trunk passed
+1 💚	shadedclient	20m 33s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 37s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 37s		the patch passed
+1 💚	compile	0m 33s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 30s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 130 unchanged - 0 fixed = 139 total (was 130)
+1 💚	mvnsite	0m 40s		the patch passed
+1 💚	javadoc	0m 32s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 58s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 45s		the patch passed
+1 💚	shadedclient	20m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	183m 6s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		269m 55s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestDirectoryScanner

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/1/artifact/out/Dockerfile
GITHUB PR	#6343
JIRA Issue	HDFS-16083
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 6be53f9fa535 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a50b726`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/1/testReport/
Max. process+thread count	4591 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ThinkerLei · 2023-12-12T02:30:58Z

@hadoop-yetus

tomscut · 2023-12-12T03:20:18Z

Thanks for reporting this. We talked about this before, see #4087.

ThinkerLei · 2023-12-12T03:43:12Z

Thanks for reporting this. We talked about this before, see #4087.

It's very funny. Please see HDFS-16083

hadoop-yetus · 2023-12-12T06:52:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 2s		trunk passed
+1 💚	compile	0m 40s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 35s		trunk passed
+1 💚	mvnsite	0m 43s		trunk passed
+1 💚	javadoc	0m 38s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 4s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 45s		trunk passed
+1 💚	shadedclient	22m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	compile	0m 37s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 37s		the patch passed
+1 💚	compile	0m 34s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 28s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 130 unchanged - 0 fixed = 139 total (was 130)
+1 💚	mvnsite	0m 36s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 57s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 41s		the patch passed
+1 💚	shadedclient	23m 23s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	167m 40s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	0m 25s	/results-asflicense.txt	The patch generated 193 ASF License warnings.
		261m 39s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestStoragePolicySummary
	hadoop.hdfs.server.datanode.TestDirectoryScanner
	hadoop.hdfs.TestDFSStripedInputStream
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestReencryptionWithKMS
	hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot
	hadoop.hdfs.TestEncryptionZonesWithKMS
	hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.hdfs.server.namenode.TestReencryptionHandler
	hadoop.hdfs.server.namenode.TestAddStripedBlocks
	hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics
	hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.TestAddBlock
	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.server.namenode.TestUpgradeDomainBlockPlacementPolicy
	hadoop.hdfs.TestDFSRollback

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/3/artifact/out/Dockerfile
GITHUB PR	#6343
JIRA Issue	HDFS-16083
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux ea65c3a6f22a 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `08da854`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/3/testReport/
Max. process+thread count	4212 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-12-12T07:22:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 39s		trunk passed
+1 💚	compile	0m 38s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 37s		trunk passed
+1 💚	mvnsite	0m 40s		trunk passed
+1 💚	javadoc	0m 39s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 2s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 42s		trunk passed
+1 💚	shadedclient	21m 51s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 36s		the patch passed
+1 💚	compile	0m 37s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 37s		the patch passed
+1 💚	compile	0m 31s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 31s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 29s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 130 unchanged - 0 fixed = 139 total (was 130)
+1 💚	mvnsite	0m 38s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 57s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 43s		the patch passed
+1 💚	shadedclient	22m 45s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	211m 28s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+0 🆗	asflicense	0m 27s		ASF License check generated no output?
		302m 44s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.server.namenode.snapshot.TestFsShellMoveToTrashWithSnapshots
	hadoop.hdfs.server.blockmanagement.TestBlockManager
	hadoop.hdfs.server.blockmanagement.TestErasureCodingCorruption
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.hdfs.TestDFSStripedInputStream
	hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
	hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
	hadoop.hdfs.server.datanode.TestDataNodeReconfiguration
	hadoop.hdfs.TestDFSShell
	hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling
	hadoop.hdfs.server.blockmanagement.TestStorageBlockPoolUsageStdDev
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.server.blockmanagement.TestBlockReportRateLimiting
	hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
	hadoop.hdfs.TestErasureCodingMultipleRacks
	hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/2/artifact/out/Dockerfile
GITHUB PR	#6343
JIRA Issue	HDFS-16083
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux dd8de7528143 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `f701eb2`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/2/testReport/
Max. process+thread count	5040 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

…ts log

hadoop-yetus · 2023-12-12T16:54:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 46s		trunk passed
+1 💚	compile	0m 43s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 40s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 38s		trunk passed
+1 💚	mvnsite	0m 43s		trunk passed
+1 💚	javadoc	0m 41s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 7s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 44s		trunk passed
+1 💚	shadedclient	20m 10s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 35s		the patch passed
+1 💚	compile	0m 38s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 38s		the patch passed
+1 💚	compile	0m 34s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 29s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 9 new + 130 unchanged - 0 fixed = 139 total (was 130)
+1 💚	mvnsite	0m 34s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 58s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 44s		the patch passed
+1 💚	shadedclient	20m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	183m 0s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		268m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/4/artifact/out/Dockerfile
GITHUB PR	#6343
JIRA Issue	HDFS-16083
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 5c16a8ff4f62 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `9a3d123`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/4/testReport/
Max. process+thread count	3996 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6343/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

github-actions bot added HDFS trunk labels Dec 11, 2023

ThinkerLei force-pushed the fix/HDFS-16083 branch 2 times, most recently from f701eb2 to 08da854 Compare December 12, 2023 02:30

HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi…

9a3d123

…ts log

ThinkerLei force-pushed the fix/HDFS-16083 branch from 08da854 to 9a3d123 Compare December 12, 2023 12:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi… #6343

HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi… #6343

Uh oh!

ThinkerLei commented Dec 11, 2023

Uh oh!

hadoop-yetus commented Dec 11, 2023

Uh oh!

ThinkerLei commented Dec 12, 2023

Uh oh!

tomscut commented Dec 12, 2023

Uh oh!

ThinkerLei commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

Uh oh!

HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi… #6343

Are you sure you want to change the base?

HDFS-16083: Forbid Observer NameNode trigger active namenode roll edi… #6343

Uh oh!

Conversation

ThinkerLei commented Dec 11, 2023

Uh oh!

hadoop-yetus commented Dec 11, 2023

Uh oh!

ThinkerLei commented Dec 12, 2023

Uh oh!

tomscut commented Dec 12, 2023

Uh oh!

ThinkerLei commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

hadoop-yetus commented Dec 12, 2023

Uh oh!

Uh oh!