HADOOP-19538. Update Boost to 1.86.0 in Windows build image #7601

stoty · 2025-04-11T08:52:28Z

Description of PR

Update Boost to 1.86.0 in the Windows Build image

How was this patch tested?

Built Hadoop in the modified Docker image

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

stoty · 2025-04-11T08:52:49Z

FYI @GauthamBanasandra

GauthamBanasandra · 2025-04-11T09:29:00Z

Thanks a lot for the fix @stoty. I'll start a run on the Windows Jenkins CI for Hadoop and ping here.

hadoop-yetus · 2025-04-11T09:33:48Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/2/console in case of problems.

GauthamBanasandra · 2025-04-11T09:35:20Z

I've started the Jenkins CI run for Windows - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/878/console.
Let's see how this goes.

hadoop-yetus · 2025-04-11T10:50:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 15s		Maven dependency ordering for branch
+1 💚	shadedclient	41m 5s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for patch
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	shadedclient	33m 4s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	asflicense	0m 37s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		77m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/2/artifact/out/Dockerfile
GITHUB PR	#7601
Optional Tests	dupname asflicense codespell detsecrets jsonlint
uname	Linux 94b94add86b5 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5ec3dc2`
Max. process+thread count	703 (vs. ulimit of 5500)
modules	C: U:
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

stoty · 2025-04-11T12:11:16Z

I've tried to add an "_asflicense": field like some other json files have in Hadoop, but vcpkg won't accept it. @GauthamBanasandra .

GauthamBanasandra · 2025-04-12T05:27:48Z

@stoty could you please exclude this JSON file from the license check by adding its path here?

hadoop/pom.xml

Line 575 in f097d68

That should fix the issue.

hadoop-yetus · 2025-04-14T05:08:17Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/3/console in case of problems.

stoty · 2025-04-14T05:09:27Z

Done, @GauthamBanasandra .
I've also moved vcpkg.json into a subdirectory, to avoid cluttering the Dockerfile directory.

hadoop-yetus · 2025-04-14T05:09:53Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/4/console in case of problems.

stoty · 2025-04-14T06:14:49Z

Re-started the windows CI:
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/882/

GauthamBanasandra

Thanks @stoty. The changes LGTM. I'll merge this after the Windows CI finishes.

hadoop-yetus · 2025-04-14T20:18:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	20m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 17s		trunk passed
+1 💚	compile	15m 51s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	13m 35s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	20m 20s		trunk passed
+1 💚	javadoc	9m 37s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 40s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	137m 21s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	137m 42s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	30m 13s		the patch passed
+1 💚	compile	15m 0s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	15m 0s		the patch passed
+1 💚	compile	13m 33s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	13m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	18m 22s		the patch passed
+1 💚	javadoc	9m 28s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 40s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	70m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	620m 40s	/patch-unit-root.txt	root in the patch passed.
-1 ❌	asflicense	1m 25s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		911m 44s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/3/artifact/out/Dockerfile
GITHUB PR	#7601
Optional Tests	dupname asflicense codespell detsecrets jsonlint compile javac javadoc mvninstall mvnsite unit shadedclient xmllint
uname	Linux 069d96a89819 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0758cd6`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/3/testReport/
Max. process+thread count	3622 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-04-14T20:36:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	20m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 57s		trunk passed
+1 💚	compile	17m 28s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	14m 51s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	21m 20s		trunk passed
+1 💚	javadoc	10m 6s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 32s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	151m 11s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	151m 33s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	34m 38s		the patch passed
+1 💚	compile	16m 50s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	16m 50s		the patch passed
+1 💚	compile	15m 2s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	15m 2s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 43s		the patch passed
+1 💚	javadoc	10m 1s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 29s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	78m 26s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	607m 32s	/patch-unit-root.txt	root in the patch passed.
-1 ❌	asflicense	1m 27s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		927m 50s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/4/artifact/out/Dockerfile
GITHUB PR	#7601
Optional Tests	dupname asflicense codespell detsecrets jsonlint compile javac javadoc mvninstall mvnsite unit shadedclient xmllint
uname	Linux 7039507a8cf0 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0758cd6`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/4/testReport/
Max. process+thread count	3296 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

pom.xml

hadoop-yetus · 2025-04-15T05:12:17Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/5/console in case of problems.

GauthamBanasandra · 2025-04-15T13:23:25Z

Thanks @stoty. I've started another run on Windows - https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-win10-x86_64/884/.

hadoop-yetus · 2025-04-15T18:17:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 3s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	jsonlint	0m 0s		jsonlint was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 57s		trunk passed
+1 💚	compile	15m 43s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	13m 37s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	20m 58s		trunk passed
+1 💚	javadoc	9m 35s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 42s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	138m 47s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	139m 7s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	31m 19s		the patch passed
+1 💚	compile	15m 7s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	15m 7s		the patch passed
+1 💚	compile	13m 48s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	13m 48s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	19m 18s		the patch passed
+1 💚	javadoc	9m 28s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 39s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	71m 31s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	512m 5s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	0m 58s		The patch does not generate ASF License warnings.
		786m 27s

Reason	Tests
Unreaped Processes	root:2
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/5/artifact/out/Dockerfile
GITHUB PR	#7601
Optional Tests	dupname asflicense codespell detsecrets jsonlint compile javac javadoc mvninstall mvnsite unit shadedclient xmllint
uname	Linux 7516e99b1b01 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e6a3134`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Unreaped Processes Log	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/5/artifact/out/patch-unit-root-reaper.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/5/testReport/
Max. process+thread count	3394 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7601/5/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

GauthamBanasandra · 2025-04-17T17:46:46Z

Compilation is passing now on Windows -

Logs from Jenkins CI -
archive.zip

Thanks for fixing this @stoty.

HADOOP-19538. Update Boost to 1.86.0 in Windows build image

5ec3dc2

github-actions bot added trunk Infra labels Apr 11, 2025

stoty closed this Apr 11, 2025

stoty reopened this Apr 11, 2025

GauthamBanasandra assigned stoty Apr 11, 2025

GauthamBanasandra self-requested a review April 11, 2025 09:29

github-actions bot added the build label Apr 14, 2025

add vcpkg.json to the RAT exclude list, and move it into a subdirectory

0758cd6

stoty force-pushed the HADOOP-19538 branch from 97e912a to 0758cd6 Compare April 14, 2025 05:07

GauthamBanasandra approved these changes Apr 14, 2025

View reviewed changes

ayushtkn reviewed Apr 14, 2025

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

fix rat exclude path

e6a3134

GauthamBanasandra merged commit 263404e into apache:trunk Apr 17, 2025
1 of 4 checks passed

GauthamBanasandra mentioned this pull request May 4, 2025

HADOOP-19560. Update build instructions for Windows #7673

Merged

4 tasks

HADOOP-19538. Update Boost to 1.86.0 in Windows build image #7601

HADOOP-19538. Update Boost to 1.86.0 in Windows build image #7601

Uh oh!

Conversation

stoty commented Apr 11, 2025

Description of PR

How was this patch tested?

For code changes:

Uh oh!

stoty commented Apr 11, 2025

Uh oh!

GauthamBanasandra commented Apr 11, 2025

Uh oh!

hadoop-yetus commented Apr 11, 2025

Uh oh!

GauthamBanasandra commented Apr 11, 2025

Uh oh!

hadoop-yetus commented Apr 11, 2025

Uh oh!

stoty commented Apr 11, 2025

Uh oh!

GauthamBanasandra commented Apr 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Apr 14, 2025

Uh oh!

stoty commented Apr 14, 2025

Uh oh!

hadoop-yetus commented Apr 14, 2025

Uh oh!

stoty commented Apr 14, 2025

Uh oh!

GauthamBanasandra left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Apr 14, 2025

Uh oh!

hadoop-yetus commented Apr 14, 2025

Uh oh!

Uh oh!

hadoop-yetus commented Apr 15, 2025

Uh oh!

GauthamBanasandra commented Apr 15, 2025

Uh oh!

hadoop-yetus commented Apr 15, 2025

Uh oh!

GauthamBanasandra commented Apr 17, 2025

Uh oh!

Uh oh!

Uh oh!

GauthamBanasandra commented Apr 12, 2025 •

edited

Loading