HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map #6547

srinireddy2020 · 2024-12-16T09:55:06Z

CellSortImporter emits values wrapped inside MapReduceExtendedCell
Converted KV to CellUtil.

srinireddy2020 · 2024-12-16T11:19:30Z

@anoopsjohn
As per your suggestion #3232 (comment)
I have changed and tested in the branch-2. It is working as expected.

Please review the changes.

pankaj72981 · 2024-12-19T17:46:14Z

hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java

@@ -413,7 +417,7 @@ public void map(ImmutableBytesWritable row, Result value, Context context) throw
            // skip if we filtered it out
            if (kv == null) continue;
            Cell ret = convertKv(kv, cfRenameMap);
-            context.write(new CellWritableComparable(ret), ret);
+            context.write(new CellWritableComparable(ret), new MapReduceExtendedCell(ret));
          }


Can you please add UT?

Thanks for looking into this issue.
Comment addressed, please verify.

Apache9 · 2024-12-29T13:35:33Z

We do not have this problem on master branch?

Apache9 · 2025-01-13T13:59:31Z

Any updates here?

srinireddy2020 · 2025-01-13T17:06:16Z

Any updates here?

Sorry for the Late reply, This issue is exist in the master branch also.

srinireddy2020 · 2025-01-14T15:10:39Z

We do not have this problem on master branch?

Raised PR for the Master Branch.
@Apache9 please check and review.

… in value from map

Apache-HBase · 2025-03-10T21:29:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 57s		branch-2 passed
+1 💚	compile	0m 59s		branch-2 passed
+1 💚	checkstyle	0m 23s		branch-2 passed
+1 💚	spotbugs	0m 57s		branch-2 passed
+1 💚	spotless	1m 17s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 43s		the patch passed
+1 💚	compile	0m 48s		the patch passed
+1 💚	javac	0m 48s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 16s		the patch passed
+1 💚	spotbugs	0m 54s		the patch passed
+1 💚	hadoopcheck	21m 45s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	1m 12s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 17s		The patch does not generate ASF License warnings.
		41m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6547
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 91581a226abb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5908276`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	76 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-03-10T21:40:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 34s		branch-2 passed
+1 💚	compile	0m 17s		branch-2 passed
+1 💚	javadoc	0m 13s		branch-2 passed
+1 💚	shadedjars	5m 18s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 23s		the patch passed
+1 💚	compile	0m 17s		the patch passed
+1 💚	javac	0m 17s		the patch passed
+1 💚	javadoc	0m 12s		the patch passed
+1 💚	shadedjars	5m 18s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	13m 43s		hbase-mapreduce in the patch passed.
		32m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6547
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux adbe73f0aa75 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5908276`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/testReport/
Max. process+thread count	2625 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-03-10T22:19:53Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 4s		Docker mode activated.
-0 ⚠️	yetus	0m 7s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 22s		branch-2 passed
+1 💚	compile	0m 34s		branch-2 passed
+1 💚	javadoc	0m 23s		branch-2 passed
+1 💚	shadedjars	8m 12s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 30s		the patch passed
+1 💚	compile	0m 31s		the patch passed
+1 💚	javac	0m 31s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed
+1 💚	shadedjars	8m 10s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	19m 28s		hbase-mapreduce in the patch passed.
		48m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6547
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 2f763ad49cd9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5908276`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/testReport/
Max. process+thread count	2876 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-03-10T22:20:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 11s		Docker mode activated.
-0 ⚠️	yetus	0m 7s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 31s		branch-2 passed
+1 💚	compile	0m 25s		branch-2 passed
+1 💚	javadoc	0m 17s		branch-2 passed
+1 💚	shadedjars	7m 56s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 16s		the patch passed
+1 💚	compile	0m 37s		the patch passed
+1 💚	javac	0m 37s		the patch passed
+1 💚	javadoc	0m 23s		the patch passed
+1 💚	shadedjars	8m 5s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	24m 15s		hbase-mapreduce in the patch passed.
		53m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6547
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 02c230b6ebf0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5908276`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/testReport/
Max. process+thread count	2767 (vs. ulimit of 30000)
modules	C: hbase-mapreduce U: hbase-mapreduce
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/6/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

… in value from map (#6547) Signed-off-by: Pankaj Kumar<pankajkumar@apache.org> (cherry picked from commit 3573930)

This comment has been minimized.

Sign in to view

pankaj72981 reviewed Dec 19, 2024

View reviewed changes

srinireddy2020 force-pushed the HBASE-25839 branch from 53be03d to 032b99e Compare December 19, 2024 19:25

This comment has been minimized.

Sign in to view

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch…

5908276

… in value from map

srinireddy2020 force-pushed the HBASE-25839 branch from 032b99e to 5908276 Compare January 15, 2025 03:10

This comment has been minimized.

Sign in to view

pankaj72981 approved these changes Mar 12, 2025

View reviewed changes

pankaj72981 merged commit 3573930 into apache:branch-2 Mar 12, 2025
1 check passed

pankaj72981 pushed a commit that referenced this pull request Mar 12, 2025

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch…

9f964d4

… in value from map (#6547) Signed-off-by: Pankaj Kumar<pankajkumar@apache.org> (cherry picked from commit 3573930)

pankaj72981 pushed a commit that referenced this pull request Mar 12, 2025

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch…

d52ed26

… in value from map (#6547) Signed-off-by: Pankaj Kumar<pankajkumar@apache.org> (cherry picked from commit 3573930)

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map #6547

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map #6547

Uh oh!

Conversation

srinireddy2020 commented Dec 16, 2024

Uh oh!

srinireddy2020 commented Dec 16, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pankaj72981 Dec 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srinireddy2020 Dec 19, 2024

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache9 commented Dec 29, 2024

Uh oh!

Apache9 commented Jan 13, 2025

Uh oh!

srinireddy2020 commented Jan 13, 2025

Uh oh!

srinireddy2020 commented Jan 14, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Mar 10, 2025

Uh oh!

Apache-HBase commented Mar 10, 2025

Uh oh!

Apache-HBase commented Mar 10, 2025

Uh oh!

Apache-HBase commented Mar 10, 2025

Uh oh!

Uh oh!

Uh oh!

pankaj72981 Dec 19, 2024 •

edited

Loading