HADOOP-18088. Replace log4j 1.x with reload4j. #4052
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Tarball built by Running tutorial code on |
|
💔 -1 overall
This message was automatically generated. |
iwasakims
force-pushed
the
HADOOP-18088-3.3
branch
from
803bef4
to
aa7054b
Compare
|
💔 -1 overall
This message was automatically generated. |
jojochuang
reviewed
Mar 9, 2022
|
@jojochuang I updated the patch based on your suggestion. The tarball built by |
|
💔 -1 overall
This message was automatically generated. |
jojochuang
reviewed
Mar 11, 2022
There was a problem hiding this comment.
@Apache9 what do you think? It looks mostly okay to me but HBase fails to build because reload4j is not permitted in the HBase dependency. This PR is targeting Hadoop 3.3.3.
|
It is fine. In HBase will exclude the log4j related jars while depending on hadoop, after the change here, we just need to add the reload4j related jars to the exclusions section too. |
|
@Apache9 looking at this in 3.3.x too, FYI |
steveloughran
requested changes
Apr 4, 2022
| @@ -151,7 +151,7 @@ | |||
| <!-- Leave commons-logging unshaded so downstream users can configure logging. --> | |||
| <exclude>commons-logging:commons-logging</exclude> | |||
| <!-- Leave log4j unshaded so downstream users can configure logging. --> | |||
There was a problem hiding this comment.
nit: comment needs updating
| @@ -85,7 +85,7 @@ | |||
| <!-- Leave commons-logging unshaded so downstream users can configure logging. --> | |||
| <exclude>commons-logging:commons-logging</exclude> | |||
| <!-- Leave log4j unshaded so downstream users can configure logging. --> | |||
There was a problem hiding this comment.
nit: comment needs updating
There was a problem hiding this comment.
I updated the comments. Thanks, @steveloughran.
|
💔 -1 overall
This message was automatically generated. |
Co-authored-by: Masatake Iwasaki <iwasakims@apache.org>
…f hadoop-project POM. Co-authored-by: Masatake Iwasaki <iwasakims@apache.org>
iwasakims
force-pushed
the
HADOOP-18088-3.3
branch
from
3b3eadf
to
a375ff4
Compare
|
I found that we need to update assembly definition under hadoop-assemblies too. I filed HADOOP-18192(#4136) for branch-3.2. I added equivalent fix for branch-3.3 here in the last commit. @steveloughran |
|
|
good catch about assemblies. i think you can use findclass to identify which jar a log4j or slf4j class is loaded from,just as i have done for the 1.2 classes |
|
Apache9
approved these changes
Apr 5, 2022
| @@ -89,7 +89,7 @@ | |||
| <!-- Leave commons-logging unshaded so downstream users can configure logging. --> | |||
| <exclude>commons-logging:commons-logging</exclude> | |||
| <!-- Leave log4j unshaded so downstream users can configure logging. --> | |||
hadoop-project/pom.xml
Outdated
| @@ -81,8 +81,8 @@ | |||
| <httpcore.version>4.4.13</httpcore.version> | |||
|
|
|||
| <!-- SLF4J/LOG4J version --> | |||
| <slf4j.version>1.7.30</slf4j.version> | |||
| <log4j.version>1.2.17</log4j.version> | |||
| <slf4j.version>1.7.35</slf4j.version> | |||
There was a problem hiding this comment.
The diff from 1.7.35 looks safe and good to upgrade here. I bumped the version.
2022-02-08 - Release of SLF4J 1.7.36
Correct corrupt "Export-Package" declaration in MANIFEST.MF in log4j-over-slf4j module. This fixes SLF4J-541 reported by Flavio Donzé.Starting with version 1.7.36, slf4j releases will be reproducible. By reproducible we mean that anyone checking out the code corresponding to the release version from source code repository and building that local copy, will obtain an identical binary to the published binary.
| @@ -2167,6 +2269,9 @@ | |||
| <exclude>com.sun.jersey.jersey-test-framework:*</exclude> | |||
| <exclude>com.google.inject:guice</exclude> | |||
| <exclude>org.ow2.asm:asm</exclude> | |||
|
|
|||
| <exclude>org.slf4j:slf4j-log4j12</exclude> | |||
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
are the tests just HDFS playing up? if so, I'm not worried, though I'd hope those test gets fixed. |
|
I can reproduce the failure of TestOfflineImageViewer#testReverseXmlWithoutSnapshotDiffSection, TestRouterRpc#testSetBalancerBandwidth and TestRouterRpcMultiDestination#testSetBalancerBandwidth on my local even without the patch. It should be addressed in another JIRA. |
|
Ok +1 from me |
steveloughran
approved these changes
Apr 6, 2022
|
I merged this. Thanks, @steveloughran, @jojochuang and @Apache9. |
steveloughran
pushed a commit
to steveloughran/hadoop
that referenced
this pull request
Apr 7, 2022
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
steveloughran
pushed a commit
to steveloughran/hadoop
that referenced
this pull request
Apr 12, 2022
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
asfgit
pushed a commit
that referenced
this pull request
Apr 13, 2022
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
asfgit
pushed a commit
that referenced
this pull request
Apr 14, 2022
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
ayushtkn
reviewed
May 7, 2022
Comment on lines
1688
to
+1702
| <groupId>org.apache.hbase</groupId> | ||
| <artifactId>hbase-server</artifactId> | ||
| <version>${hbase.version}</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>log4j</groupId> | ||
| <artifactId>log4j</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.hbase</groupId> | ||
| <artifactId>hbase-server</artifactId> | ||
| <version>${hbase.version}</version> | ||
| <scope>test</scope> |
There was a problem hiding this comment.
@iwasakims hbase-server was already there, with compile scope, why was it added here again with test scope?
It seems to give warnings while building 3.3.3 to me
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-project:pom:3.3.3
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.apache.hbase:hbase-server:jar -> duplicate declaration of version ${hbase.version} @ line 1691, column 19
steveloughran
pushed a commit
to steveloughran/hadoop
that referenced
this pull request
Jun 3, 2022
Change-Id: Id2f25f0b9d26e0ff73f828acbfb4e6fbac3425cc Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
steveloughran
pushed a commit
to steveloughran/hadoop
that referenced
this pull request
Feb 12, 2024
Change-Id: I7aa29cdbe66e3eabcff5d30f244c55563a6dfa7f Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
4 tasks
steveloughran
added a commit
that referenced
this pull request
Feb 13, 2024
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org> Includes HADOOP-18354. Upgrade reload4j to 1.22.2 due to XXE vulnerability (#4607). Log4j 1.2.17 has been replaced by reloadj 1.22.2 SLF4J is at 1.7.36
iwasakims
pushed a commit
that referenced
this pull request
Feb 16, 2024
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org> Includes HADOOP-18354. Upgrade reload4j to 1.22.2 due to XXE vulnerability (#4607). Log4j 1.2.17 has been replaced by reloadj 1.22.2 SLF4J is at 1.7.36 (cherry picked from commit 095dfcc)
slfan1989
pushed a commit
that referenced
this pull request
Mar 3, 2024
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org> Includes HADOOP-18354. Upgrade reload4j to 1.22.2 due to XXE vulnerability (#4607). Log4j 1.2.17 has been replaced by reloadj 1.22.2 SLF4J is at 1.7.36 (cherry picked from commit 095dfcc)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.