HADOOP-17727. Modularize docker images #3043
Conversation
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
GauthamBanasandra
force-pushed
the
modularize-dockerfiles
branch
from
1e57249 to
c20a85a
Compare
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
GauthamBanasandra
force-pushed
the
modularize-dockerfiles
branch
from
ff5db72 to
d41c011
Compare
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
(!) A patch to the testing environment has been detected. |
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
* Replaced explicit installation of packages with script invocations.
* Replaced explicit installations of packages with script invocations.
* Added a readme file for describing the package management module.
* Added the necessary environment variables and install scripts.
* PROTOBUF_HOME was commented out. Fixed it.
GauthamBanasandra
force-pushed
the
modularize-dockerfiles
branch
from
08571af to
1496a6e
Compare
|
(!) A patch to the testing environment has been detected. |
|
(!) A patch to the testing environment has been detected. |
* Added ASF license header for README.md in dev-support/docker.
GauthamBanasandra
force-pushed
the
modularize-dockerfiles
branch
from
7c6037f to
f6433cc
Compare
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
@jojochuang the unit tests get stuck when I add the json files to be excluded from RAT check in pom.xml (I don't know why though, but I've confirmed this multiple times). Hence, I've removed them and the CI run is complete as posted in the YETUS comment above. |
There was a problem hiding this comment.
I suspect we also need to update the create-release script.
Ran dev-support/bin/create-release --asfrelease --docker --dockercache --native on a Ubuntu ARM, and it failed with this error:
Step 8/34 : COPY pkg-resolver pkg-resolver
COPY failed: file not found in build context or excluded by .dockerignore: stat pkg-resolver: file does not exist
$ docker run -i -t --privileged -v /home/ubuntu/.gnupg:/home/ubuntu/.gnupg -v /home/ubuntu/hadoop/patchprocess:/home/ubuntu/hadoop/patchprocess -v /home/ubuntu/hadoop/target/artifacts:/home/ubuntu/hadoop/target/artifacts -v /home/ubuntu/.m2/settings.xml:/home/ubuntu/.m2/settings.xml -v /home/ubuntu/hadoop:/build/source -u ubuntu -w /build/source hadoop/createrelease:3.4.0-SNAPSHOT_25570 /build/source/dev-support/bin/create-release --deploy --mvncache=/maven --asfrelease --dockercache --native --indocker
Unable to find image 'hadoop/createrelease:3.4.0-SNAPSHOT_25570' locally
docker: Error response from daemon: pull access denied for hadoop/createrelease, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
Guess the script needs to add an additional mount volume?
|
That being said, given that Inigo gave a +1 and we went through several iterations, I don't mind to fix that in a separate jira. |
|
(!) A patch to the testing environment has been detected. |
|
@jojochuang thanks for the review.
We can't use a mount volume here since the
I've fixed it in this PR by providing the build context path during docker build. Please see my changes in |
* create-release script was failing since it couldn't find pkg-resolver directory. * Fixed it by specifying the path to the build context.
GauthamBanasandra
force-pushed
the
modularize-dockerfiles
branch
from
c65107c to
263666d
Compare
|
(!) A patch to the testing environment has been detected. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@jojochuang could you please approve this PR if it looks alright? I've fixed the issue with create-release as well. |
|
Thanks a lot for your reviews and suggestions @goiri , @jojochuang |
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
the package dependencies needed
for building Hadoop on the various
platforms.
the dependencies on the all the
different platforms for which we
support building Hadoop on.