Skip to content

Fix NPE in Find.java #3164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: trunk
Choose a base branch
from

Conversation

containerAnalyzer
Copy link

Hello,
Our static analyzer found a following potential NPE. We have checked the feasibility of this execution trace. It is necessary to defend this vulnerability to improve the code quality. We have provided the patch for you. Please check and confirm it.

Here is the bug trace.

  1. Select the false branch at this point (expressionClass==null is true), and null assigned to instance

    Expression instance = null;
    if (expressionClass != null) {
    instance = ReflectionUtils.newInstance(expressionClass, conf);
    }
    return instance;

  2. Return instance to caller, which can be null (The return value can be null)

  3. Function createExpression executes and stores the return value to expr (expr can be null)

    Expression expr = factory.createExpression(exprClass, null);

  4. Function add executes and primaries contains null value

  5. Function next executes and stores the return value to expr (expr can be null)

  6. expr is passed as the this pointer to function getUsage (expr can be null), which will leak to null pointer dereference

Commit: 986d0a4

ContainerAnalyzer

@containerAnalyzer
Copy link
Author

Hello,
There is another NPE in DancingLinks.java. The patch is also provided in the pr. Here is the bug trace.

  1. Return null to caller

  2. Function advance executes and returns

  3. Function add executes and choices contains null value

  4. Function get executes and returns

  5. The return value of function get is used as the 1st parameter in function rollback (the return value of function get can be null)

  6. Get the value of row.left, which will leak to null pointer dereference

Commit: 986d0a4

ContainerAnalyzer

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 39s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 30m 56s trunk passed
+1 💚 compile 23m 55s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 18m 34s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 1m 8s trunk passed
+1 💚 mvnsite 1m 36s trunk passed
+1 💚 javadoc 1m 8s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 1m 41s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 2m 25s trunk passed
+1 💚 shadedclient 15m 40s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 54s the patch passed
+1 💚 compile 21m 12s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 21m 12s the patch passed
+1 💚 compile 19m 2s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 19m 2s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 3s the patch passed
+1 💚 mvnsite 1m 33s the patch passed
+1 💚 javadoc 1m 1s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 1m 38s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌ spotbugs 2m 35s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 16m 2s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 10s hadoop-common in the patch passed.
+1 💚 asflicense 0m 49s The patch does not generate ASF License warnings.
181m 14s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Nullcheck of expr at line 114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:[line 114]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/artifact/out/Dockerfile
GITHUB PR #3164
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux d0cda20ee375 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / c905fab
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/testReport/
Max. process+thread count 1310 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 54s Maven dependency ordering for branch
+1 💚 mvninstall 20m 8s trunk passed
+1 💚 compile 21m 16s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 18m 20s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 53s trunk passed
+1 💚 mvnsite 2m 27s trunk passed
+1 💚 javadoc 1m 56s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 2m 23s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 3m 28s trunk passed
+1 💚 shadedclient 14m 15s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 1m 18s the patch passed
+1 💚 compile 20m 26s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 20m 26s the patch passed
+1 💚 compile 18m 18s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 18m 18s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 40s the patch passed
+1 💚 mvnsite 2m 25s the patch passed
+1 💚 javadoc 1m 54s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 2m 27s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌ spotbugs 2m 36s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 14m 38s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 2s hadoop-common in the patch passed.
+1 💚 unit 1m 2s hadoop-mapreduce-examples in the patch passed.
+1 💚 asflicense 0m 59s The patch does not generate ASF License warnings.
194m 12s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Nullcheck of expr at line 114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:[line 114]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/2/artifact/out/Dockerfile
GITHUB PR #3164
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 6d045ac9e9e0 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / dc98af8
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/2/testReport/
Max. process+thread count 1500 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-examples U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@containerAnalyzer
Copy link
Author

containerAnalyzer commented Jul 7, 2021

This is another NPE occurring in DumpS3GuardDynamoTable.java. The patch has been submitted.

  1. Return null to caller

  2. Return the return value of function getDirListingMetadataFromDirMetaAndList to caller

    return getDirListingMetadataFromDirMetaAndList(path, metas,
    get(path));

  3. Function listChildren executes and returns the null value, which is added to the list childMD.

    childMD.add(getStore().listChildren(
    childDir.getFileStatus().getPath()));

  4. The value childMD is passed as the 2nd parameter of pushAll, and it contains null value.

  5. The list reversed contains the null value after being assigned by the return value of the function reverse.

  6. The return value of function iterator is passed as the this pointer to function hasNext (the return value of function iterator can be null), which will leak to null pointer dereference

Commit: 986d0a4

ContainerAnalyzer

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 12m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 42s Maven dependency ordering for branch
+1 💚 mvninstall 20m 31s trunk passed
+1 💚 compile 21m 23s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 18m 26s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 40s trunk passed
+1 💚 mvnsite 3m 11s trunk passed
+1 💚 javadoc 2m 20s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 2m 57s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 4m 42s trunk passed
+1 💚 shadedclient 14m 27s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 1m 54s the patch passed
+1 💚 compile 20m 41s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 20m 41s the patch passed
+1 💚 compile 18m 33s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 18m 33s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 3m 39s /results-checkstyle-root.txt root: The patch generated 1 new + 21 unchanged - 0 fixed = 22 total (was 21)
+1 💚 mvnsite 3m 10s the patch passed
+1 💚 javadoc 2m 16s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 2m 59s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌ spotbugs 2m 37s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 14m 36s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 16m 56s hadoop-common in the patch passed.
+1 💚 unit 0m 57s hadoop-mapreduce-examples in the patch passed.
+1 💚 unit 2m 18s hadoop-aws in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
215m 32s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Nullcheck of expr at line 114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:[line 114]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/3/artifact/out/Dockerfile
GITHUB PR #3164
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 99f4d797adb0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / c470ce6
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/3/testReport/
Max. process+thread count 3158 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-aws U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 1s Maven dependency ordering for branch
+1 💚 mvninstall 21m 52s trunk passed
+1 💚 compile 22m 17s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 19m 27s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 58s trunk passed
+1 💚 mvnsite 3m 9s trunk passed
+1 💚 javadoc 2m 25s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 3m 7s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 4m 53s trunk passed
+1 💚 shadedclient 20m 26s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for patch
+1 💚 mvninstall 2m 5s the patch passed
+1 💚 compile 22m 0s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 22m 0s the patch passed
+1 💚 compile 19m 12s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 19m 12s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 3m 41s /results-checkstyle-root.txt root: The patch generated 1 new + 20 unchanged - 0 fixed = 21 total (was 20)
+1 💚 mvnsite 3m 22s the patch passed
+1 💚 javadoc 2m 34s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 3m 10s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌ spotbugs 2m 36s /new-spotbugs-hadoop-common-project_hadoop-common.html hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚 shadedclient 20m 39s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 18s hadoop-common in the patch passed.
+1 💚 unit 1m 1s hadoop-mapreduce-examples in the patch passed.
+1 💚 unit 2m 28s hadoop-aws in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
223m 35s
Reason Tests
SpotBugs module:hadoop-common-project/hadoop-common
Nullcheck of expr at line 114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:114 of value previously dereferenced in org.apache.hadoop.fs.shell.find.Find.buildDescription(ExpressionFactory) At Find.java:[line 114]
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/artifact/out/Dockerfile
GITHUB PR #3164
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 6778402f7cc1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / c470ce6
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/testReport/
Max. process+thread count 1362 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-aws U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3164/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants