Skip to content

HBASE-27904: Adding a data generator command line utility leveraging HBase bulk load #5279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: branch-2.4
Choose a base branch
from
Open

HBASE-27904: Adding a data generator command line utility leveraging HBase bulk load #5279

wants to merge 3 commits into from

Conversation

Himanshu-g81
Copy link
Contributor

@Himanshu-g81 Himanshu-g81 commented Jun 8, 2023 •
edited by virajjasani
Loading

Details in JIRA: HBASE-27904

branch-2.4 backport of #5280

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 11s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 2m 42s branch-2.4 passed
+1 💚 compile 0m 20s branch-2.4 passed
+1 💚 shadedjars 4m 25s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 0s the patch passed
+1 💚 compile 0m 17s the patch passed
+1 💚 javac 0m 17s the patch passed
+1 💚 shadedjars 4m 20s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 14s the patch passed
_ Other Tests _
-1 ❌ unit 21m 10s hbase-mapreduce in the patch failed.
38m 4s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #5279
JIRA Issue HBASE-27904
Optional Tests javac javadoc unit shadedjars compile
uname Linux dc79f87442c9 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 7d78979
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-mapreduce.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/testReport/
Max. process+thread count 2565 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 39s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 3m 57s branch-2.4 passed
+1 💚 compile 0m 55s branch-2.4 passed
+1 💚 checkstyle 0m 22s branch-2.4 passed
+1 💚 spotless 0m 59s branch has no errors when running spotless:check.
+1 💚 spotbugs 0m 38s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 28s the patch passed
+1 💚 compile 0m 56s the patch passed
-0 ⚠️ javac 0m 56s hbase-mapreduce generated 16 new + 183 unchanged - 16 fixed = 199 total (was 199)
-0 ⚠️ checkstyle 0m 15s hbase-mapreduce: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+1 💚 hadoopcheck 24m 48s Patch does not cause any errors with Hadoop 2.10.2 or 3.1.4 3.2.4 3.3.5.
+1 💚 spotless 1m 2s patch has no errors when running spotless:check.
+1 💚 spotbugs 0m 58s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
41m 49s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5279
JIRA Issue HBASE-27904
Optional Tests dupname asflicense javac hadoopcheck spotless xml compile spotbugs hbaseanti checkstyle
uname Linux eff2b75bf107 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 7d78979
Default Java Eclipse Adoptium-11.0.17+8
javac https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt
checkstyle https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 9s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 4m 19s branch-2.4 passed
+1 💚 compile 0m 26s branch-2.4 passed
+1 💚 shadedjars 6m 45s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 19s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 27s the patch passed
+1 💚 compile 0m 23s the patch passed
+1 💚 javac 0m 23s the patch passed
+1 💚 shadedjars 6m 13s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 21s the patch passed
_ Other Tests _
-1 ❌ unit 22m 49s hbase-mapreduce in the patch failed.
47m 29s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5279
JIRA Issue HBASE-27904
Optional Tests javac javadoc unit shadedjars compile
uname Linux a4ae478b737d 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 7d78979
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/testReport/
Max. process+thread count 2999 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani virajjasani self-requested a review June 8, 2023 20:05
@virajjasani virajjasani added the backport This PR is a back port of some issue or issues already committed to master label Jun 8, 2023
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 30s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 3m 29s branch-2.4 passed
+1 💚 compile 0m 40s branch-2.4 passed
+1 💚 checkstyle 0m 15s branch-2.4 passed
+1 💚 spotless 0m 57s branch has no errors when running spotless:check.
+1 💚 spotbugs 0m 43s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 46s the patch passed
+1 💚 compile 0m 37s the patch passed
-0 ⚠️ javac 0m 37s hbase-mapreduce generated 13 new + 186 unchanged - 13 fixed = 199 total (was 199)
-0 ⚠️ checkstyle 0m 12s hbase-mapreduce: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+1 💚 hadoopcheck 21m 38s Patch does not cause any errors with Hadoop 2.10.2 or 3.1.4 3.2.4 3.3.5.
+1 💚 spotless 1m 9s patch has no errors when running spotless:check.
+1 💚 spotbugs 1m 4s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 17s The patch does not generate ASF License warnings.
36m 54s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5279
Optional Tests dupname asflicense javac hadoopcheck spotless xml compile spotbugs hbaseanti checkstyle
uname Linux 41026b5e0393 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 13a063e
Default Java Eclipse Adoptium-11.0.17+8
javac https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-mapreduce.txt
checkstyle https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-mapreduce.txt
Max. process+thread count 79 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 37s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 2m 55s branch-2.4 passed
+1 💚 compile 0m 26s branch-2.4 passed
+1 💚 shadedjars 5m 22s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 17s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 39s the patch passed
+1 💚 compile 0m 20s the patch passed
+1 💚 javac 0m 20s the patch passed
+1 💚 shadedjars 5m 34s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 18s the patch passed
_ Other Tests _
-1 ❌ unit 22m 46s hbase-mapreduce in the patch failed.
42m 16s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5279
Optional Tests javac javadoc unit shadedjars compile
uname Linux e0141c310205 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 13a063e
Default Java Eclipse Adoptium-11.0.17+8
unit https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt
Test Results https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/testReport/
Max. process+thread count 3054 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 1s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 3m 6s branch-2.4 passed
+1 💚 compile 0m 22s branch-2.4 passed
+1 💚 shadedjars 4m 49s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 15s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 13s the patch passed
+1 💚 compile 0m 18s the patch passed
+1 💚 javac 0m 18s the patch passed
+1 💚 shadedjars 5m 9s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 12s the patch passed
_ Other Tests _
-1 ❌ unit 24m 20s hbase-mapreduce in the patch failed.
42m 51s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #5279
Optional Tests javac javadoc unit shadedjars compile
uname Linux f413df771a1a 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 13a063e
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-mapreduce.txt
Test Results https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/testReport/
Max. process+thread count 2610 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/hbase-kustomize-github-pr/job/PR-5279/1/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@virajjasani virajjasani Awaiting requested review from virajjasani

Assignees
No one assigned
Labels
backport This PR is a back port of some issue or issues already committed to master
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Himanshu-g81 @Apache-HBase @virajjasani