HBASE-25663 Make graceful_stop localhostname compare match even if fqdn #3048

saintstack · 2021-03-12T17:42:36Z

No description provided.

Apache-HBase · 2021-03-12T17:45:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
		_ Patch Compile Tests _
		_ Other Tests _
		1m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests
uname	Linux e8f265ad0264 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `876fec1`
Max. process+thread count	47 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-12T17:45:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
		_ Patch Compile Tests _
		_ Other Tests _
		1m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests
uname	Linux 76c0afad57fc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `876fec1`
Max. process+thread count	53 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-12T18:01:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 16s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 11s	master passed
+0 🆗	refguide	3m 43s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 50s	the patch passed
+1 💚	shellcheck	0m 1s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	3m 37s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		18m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests	dupname asflicense shellcheck shelldocs refguide
uname	Linux f7c757822cc3 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `876fec1`
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/artifact/yetus-general-check/output/branch-site/book.html
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count	66 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/1/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ekrettek

Confirmed this works when running with hostname --fqdn now!

Apache-HBase · 2021-03-15T13:30:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
		_ Patch Compile Tests _
		_ Other Tests _
		1m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests
uname	Linux 1f7e950b6142 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `aeec8ca`
Max. process+thread count	53 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T13:50:37Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 49s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 1s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 56s	master passed
+0 🆗	refguide	3m 57s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 40s	the patch passed
-0 ⚠️	shellcheck	0m 1s	The patch generated 8 new + 23 unchanged - 15 fixed = 31 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	4m 1s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		21m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests	dupname asflicense shellcheck shelldocs refguide
uname	Linux 115579193aa2 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `aeec8ca`
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/artifact/yetus-general-check/output/branch-site/book.html
shellcheck	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/artifact/yetus-general-check/output/diff-patch-shellcheck.txt
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count	65 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/2/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T15:18:06Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
		_ Patch Compile Tests _
		_ Other Tests _
		1m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests
uname	Linux 5ee6f29e63ed 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8337fb2`
Max. process+thread count	44 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T15:18:06Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
		_ Patch Compile Tests _
		_ Other Tests _
		1m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests
uname	Linux 081b2d6a3de6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8337fb2`
Max. process+thread count	47 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T15:35:40Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 43s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 15s	master passed
+0 🆗	refguide	3m 42s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 51s	the patch passed
-0 ⚠️	shellcheck	0m 1s	The patch generated 10 new + 21 unchanged - 17 fixed = 31 total (was 38)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	3m 46s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
		_ Other Tests _
+1 💚	asflicense	0m 18s	The patch does not generate ASF License warnings.
		19m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3048
Optional Tests	dupname asflicense shellcheck shelldocs refguide
uname	Linux eb68fe62b697 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8337fb2`
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-general-check/output/branch-site/book.html
shellcheck	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-general-check/output/diff-patch-shellcheck.txt
refguide	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/artifact/yetus-general-check/output/patch-site/book.html
Max. process+thread count	65 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/3/console
versions	git=2.17.1 maven=3.6.3 shellcheck=0.4.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T20:25:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#3048 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#3048
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/4/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T20:25:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#3048 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#3048
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/4/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-15T20:25:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#3048 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#3048
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3048/4/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…dn (#3048)

javierluca

@saintstack @ekrettek I am curious about how the stripServer condition in RegionMover.java will succeed after this PR:

hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/util/RegionMover.java

Line 692 in cfbae4d

if (server.getAddress().getHostName().equalsIgnoreCase(hostname) &&

From my understanding, this will compare the fqdn with "localhost", so it won't be able to strip it, leading the RegionMover to not unload and return early:

hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/util/RegionMover.java

Lines 384 to 390 in cfbae4d

    
           ServerName server = stripServer(regionServers, hostname, port); 
        
           if (server == null) { 
        
             LOG.info("Could not find server '{}:{}' in the set of region servers. giving up.", 
        
                 hostname, port); 
        
             LOG.debug("List of region servers: {}", regionServers); 
        
             return false; 
        
           }

saintstack · 2021-03-16T04:01:08Z

@javierluca Thank you for taking a look.

Looks like the stripServer needs to handle 'localhost'. Let me make an addendum....

saintstack · 2021-03-16T04:25:42Z

Or, let me just revert this and commit your patch; it works for you.

…en if fqdn (#3048)" This reverts commit f4e1ab7.

…en if fqdn (#3048)" This reverts commit 5366a7c.

javierluca · 2021-03-16T05:21:46Z

@saintstack In my opinion, your change was better than mine because we could use "localhost" as argument, and that will work very well with regionservers.sh which will bypass the ssh section when localhost, which does not work well with secured clusters:

hbase/bin/regionservers.sh

Lines 61 to 64 in 59ec375

    
           if [ "$regionservers" = "localhost" ]; then 
        
             HBASE_REGIONSERVER_ARGS="\ 
        
               -Dhbase.regionserver.port=16020 \ 
        
               -Dhbase.regionserver.info.port=16030"

Just needed to address the stripServer issue.
May be something like this will do it?

unload_hostname="$hostname"
if [[ "$hostname" -eq "localhost" ]]; then
  unload_hostname=`/bin/hostname -f`
fi
unload_args="--filename $filename --maxthreads $maxthreads $noack --operation unload \
--timeout $movetimeout --regionserverhost $unload_hostname"

If you don't mind, I would like to submit a patch including the changes of this PR + above fix for the RegionMover.

…dn (apache#3048)

javierluca · 2021-03-16T05:44:27Z

@saintstack addressing it at #3053. Please have a look 🙇

ekrettek approved these changes Mar 13, 2021

View reviewed changes

saintstack force-pushed the HBASE-25663 branch from 09e30d8 to 0753bdb Compare March 15, 2021 15:15

HBASE-25663 Make graceful_stop localhostname compare match even if fqdn

efe933a

saintstack force-pushed the HBASE-25663 branch from 0753bdb to efe933a Compare March 15, 2021 20:23

saintstack merged commit f4e1ab7 into apache:master Mar 15, 2021

saintstack added a commit that referenced this pull request Mar 15, 2021

HBASE-25663 Make graceful_stop localhostname compare match even if fq…

754300a

…dn (#3048)

saintstack added a commit that referenced this pull request Mar 15, 2021

HBASE-25663 Make graceful_stop localhostname compare match even if fq…

5366a7c

…dn (#3048)

javierluca mentioned this pull request Mar 16, 2021

HBASE-25594 graceful_stop.sh fails to unload regions when ran at localhost #2973

Closed

javierluca reviewed Mar 16, 2021

View reviewed changes

saintstack added a commit that referenced this pull request Mar 16, 2021

Revert "HBASE-25663 Make graceful_stop localhostname compare match ev…

c36e40e

…en if fqdn (#3048)" This reverts commit f4e1ab7.

saintstack added a commit that referenced this pull request Mar 16, 2021

Revert "HBASE-25663 Make graceful_stop localhostname compare match ev…

4b2cab3

…en if fqdn (#3048)" This reverts commit f4e1ab7.

saintstack added a commit that referenced this pull request Mar 16, 2021

Revert "HBASE-25663 Make graceful_stop localhostname compare match ev…

3aa7285

…en if fqdn (#3048)" This reverts commit 5366a7c.

javierluca pushed a commit to javierluca/hbase that referenced this pull request Mar 16, 2021

HBASE-25663 Make graceful_stop localhostname compare match even if fq…

b8a4a60

…dn (apache#3048)

javierluca mentioned this pull request Mar 16, 2021

HBASE-25663 Make graceful_stop localhostname compare match even if fqdn and unload regions #3053

Closed

javierluca mentioned this pull request Mar 16, 2021

HBASE-25594 Make easier to use graceful_stop on localhost mode #3054

Merged

	ServerName server = stripServer(regionServers, hostname, port);
	if (server == null) {
	LOG.info("Could not find server '{}:{}' in the set of region servers. giving up.",
	hostname, port);
	LOG.debug("List of region servers: {}", regionServers);
	return false;
	}

HBASE-25663 Make graceful_stop localhostname compare match even if fqdn #3048

HBASE-25663 Make graceful_stop localhostname compare match even if fqdn #3048

Uh oh!

Conversation

saintstack commented Mar 12, 2021

Uh oh!

Apache-HBase commented Mar 12, 2021

Uh oh!

Apache-HBase commented Mar 12, 2021

Uh oh!

Apache-HBase commented Mar 12, 2021

Uh oh!

ekrettek left a comment

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

Apache-HBase commented Mar 15, 2021

Uh oh!

javierluca left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

saintstack commented Mar 16, 2021

Uh oh!

saintstack commented Mar 16, 2021

Uh oh!

javierluca commented Mar 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

javierluca commented Mar 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

javierluca left a comment •

edited

Loading

javierluca commented Mar 16, 2021 •

edited

Loading