Skip to content

Conversation

@pyek-bot
Copy link
Collaborator

@pyek-bot pyek-bot commented Aug 5, 2025

Description

This PR copies the change from here: 21191c7

to avoid this particular CVE: https://advisories.opensearch.org/advisories/CVE-2025-48734

Original tried to fix in this PR: https://github.com/opensearch-project/ml-commons/pull/3935/files

but this doesn't seem to have worked.

As seen in the below image, these are the affected branches and 2.19 change seems to be fixed. So doing the same here.
Screenshot 2025-08-05 at 12 02 01 PM

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Pavan Yekbote <pybot@amazon.com>
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 19:06 — with GitHub Actions Error
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 19:06 — with GitHub Actions Inactive
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 19:06 — with GitHub Actions Failure
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 19:06 — with GitHub Actions Inactive
Zhangxunmt
Zhangxunmt previously approved these changes Aug 5, 2025
@dhrubo-os
Copy link
Collaborator

Have we checked the dependency insight to mauke sure this version is coming up? Also what about the algorithm module?

Can you please share the dependency insight for this beanutils in ml-commons?

@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

Have we checked the dependency insight to mauke sure this version is coming up? Also what about the algorithm module?

Can you please share the dependency insight for this beanutils in ml-commons?

I did a ./gradlew assemble, unzipped the build file and looked for the dependency there, in that I can see that this is the version 1.11.0

I ran this command for dependency insight:

./gradlew dependencyInsight --dependency commons-beanutils

This was the result:

ml-commons git:(backport_cve_fix) ./gradlew dependencyInsight --dependency commons-beanutils
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 21 (Amazon Corretto JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home
  Random Testing Seed   : C66801F4823AA9C0
  Crypto Standard       : any-supported
=======================================

> Task :dependencyInsight
No dependencies matching given input were found in configuration ':compileClasspath'

Any other way to validate?

WRT ml-algorithms, I tried to take a diff between the 2 prs, the build passed with the current change (without adding specifically to ml-algorithms) although the only usage of this dependency is in the ml-algorithms package.

Signed-off-by: Pavan Yekbote <pybot@amazon.com>
@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

I looked into the dependencies of ml-algorithms, the version comes from another dependency (opencsv) which was forcing it to 1.9, now with this explicit version, it has corrected! I can't post the entire dependencies since it is large, posting some excerpts.

|    +--- org.tribuo:tribuo-data:4.2.1
|    |    +--- org.tribuo:tribuo-core:4.2.1 (*)
|    |    +--- com.oracle.labs.olcut:olcut-core:5.2.1 (*)
|    |    +--- org.tribuo:tribuo-util-tokenization:4.2.1
|    |    |    \--- com.oracle.labs.olcut:olcut-core:5.2.1 (*)
|    |    \--- com.opencsv:opencsv:5.4
|    |         +--- org.apache.commons:commons-lang3:3.11 -> 3.12.0
|    |         +--- org.apache.commons:commons-text:1.9 -> 1.10.0 (*)
|    |         +--- commons-beanutils:commons-beanutils:1.9.4 -> 1.11.0
|    |         |    +--- commons-logging:commons-logging:1.3.5
|    |         |    \--- commons-collections:commons-collections:3.2.2
|    |         \--- org.apache.commons:commons-collections4:4.4
|    |    +--- com.fasterxml.jackson.core:jackson-databind:2.18.2 (*)
|    |    \--- com.fasterxml.jackson:jackson-bom:2.18.2 (*)
|    +--- org.apache.httpcomponents.client5:httpclient5:5.4.4 (*)
|    \--- org.apache.httpcomponents.core5:httpcore5:5.3.4
+--- commons-beanutils:commons-beanutils:1.11.0 (*)
+--- com.microsoft.onnxruntime:onnxruntime:{strictly 1.16.3} -> 1.16.3
+--- software.amazon.awssdk:bom:2.30.18
|    +--- software.amazon.awssdk:auth:2.30.18 (c)
|    +--- software.amazon.awssdk:aws-core:2.30.18 (c)
|    +--- software.amazon.awssdk:regions:2.30.18 (c)

@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

➜  ml-commons git:(backport_cve_fix) ✗ ./gradlew opensearch-ml-algorithms:dependencyInsight --dependency commons-beanutils
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 21 (Amazon Corretto JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home
  Random Testing Seed   : CCE5B42024E301DA
  Crypto Standard       : any-supported
=======================================

> Task :opensearch-ml-algorithms:dependencyInsight
commons-beanutils:commons-beanutils:1.11.0
  Variant compile:
    | Attribute Name                 | Provided | Requested    |
    |--------------------------------|----------|--------------|
    | org.gradle.status              | release  |              |
    | org.gradle.category            | library  | library      |
    | org.gradle.libraryelements     | jar      | classes      |
    | org.gradle.usage               | java-api | java-api     |
    | org.gradle.dependency.bundling |          | external     |
    | org.gradle.jvm.environment     |          | standard-jvm |
    | org.gradle.jvm.version         |          | 21           |
   Selection reasons:
      - By conflict resolution: between versions 1.11.0 and 1.9.4

commons-beanutils:commons-beanutils:1.11.0
\--- compileClasspath

commons-beanutils:commons-beanutils:1.9.4 -> 1.11.0
\--- com.opencsv:opencsv:5.4
     \--- org.tribuo:tribuo-data:4.2.1
          +--- org.tribuo:tribuo-clustering-kmeans:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-regression-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-classification-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-common-sgd:4.2.1
          |    +--- org.tribuo:tribuo-regression-sgd:4.2.1 (*)
          |    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)
          +--- org.tribuo:tribuo-anomaly-core:4.2.1
          |    \--- org.tribuo:tribuo-anomaly-libsvm:4.2.1
          |         \--- compileClasspath
          \--- org.tribuo:tribuo-common-tree:4.2.1
               \--- org.tribuo:tribuo-classification-core:4.2.1
                    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)

@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

Not sure why spotless is failing, when I run ./gradlew spotlessApply nothing changes

Signed-off-by: Pavan Yekbote <pybot@amazon.com>
@dhrubo-os
Copy link
Collaborator

➜  ml-commons git:(backport_cve_fix) ✗ ./gradlew opensearch-ml-algorithms:dependencyInsight --dependency commons-beanutils
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 21 (Amazon Corretto JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home
  Random Testing Seed   : CCE5B42024E301DA
  Crypto Standard       : any-supported
=======================================

> Task :opensearch-ml-algorithms:dependencyInsight
commons-beanutils:commons-beanutils:1.11.0
  Variant compile:
    | Attribute Name                 | Provided | Requested    |
    |--------------------------------|----------|--------------|
    | org.gradle.status              | release  |              |
    | org.gradle.category            | library  | library      |
    | org.gradle.libraryelements     | jar      | classes      |
    | org.gradle.usage               | java-api | java-api     |
    | org.gradle.dependency.bundling |          | external     |
    | org.gradle.jvm.environment     |          | standard-jvm |
    | org.gradle.jvm.version         |          | 21           |
   Selection reasons:
      - By conflict resolution: between versions 1.11.0 and 1.9.4

commons-beanutils:commons-beanutils:1.11.0
\--- compileClasspath

commons-beanutils:commons-beanutils:1.9.4 -> 1.11.0
\--- com.opencsv:opencsv:5.4
     \--- org.tribuo:tribuo-data:4.2.1
          +--- org.tribuo:tribuo-clustering-kmeans:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-regression-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-classification-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-common-sgd:4.2.1
          |    +--- org.tribuo:tribuo-regression-sgd:4.2.1 (*)
          |    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)
          +--- org.tribuo:tribuo-anomaly-core:4.2.1
          |    \--- org.tribuo:tribuo-anomaly-libsvm:4.2.1
          |         \--- compileClasspath
          \--- org.tribuo:tribuo-common-tree:4.2.1
               \--- org.tribuo:tribuo-classification-core:4.2.1
                    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)

Thanks, can you please show the same insight for plugin?

@dhrubo-os
Copy link
Collaborator

Not sure why spotless is failing, when I run ./gradlew spotlessApply nothing changes

That's weird. From the report it's showing:

Build file '/home/runner/work/ml-commons/ml-commons/plugin/build.gradle' line: 24

@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

➜  ml-commons git:(backport_cve_fix) ✗ ./gradlew opensearch-ml-algorithms:dependencyInsight --dependency commons-beanutils
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 21 (Amazon Corretto JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/amazon-corretto-21.jdk/Contents/Home
  Random Testing Seed   : CCE5B42024E301DA
  Crypto Standard       : any-supported
=======================================

> Task :opensearch-ml-algorithms:dependencyInsight
commons-beanutils:commons-beanutils:1.11.0
  Variant compile:
    | Attribute Name                 | Provided | Requested    |
    |--------------------------------|----------|--------------|
    | org.gradle.status              | release  |              |
    | org.gradle.category            | library  | library      |
    | org.gradle.libraryelements     | jar      | classes      |
    | org.gradle.usage               | java-api | java-api     |
    | org.gradle.dependency.bundling |          | external     |
    | org.gradle.jvm.environment     |          | standard-jvm |
    | org.gradle.jvm.version         |          | 21           |
   Selection reasons:
      - By conflict resolution: between versions 1.11.0 and 1.9.4

commons-beanutils:commons-beanutils:1.11.0
\--- compileClasspath

commons-beanutils:commons-beanutils:1.9.4 -> 1.11.0
\--- com.opencsv:opencsv:5.4
     \--- org.tribuo:tribuo-data:4.2.1
          +--- org.tribuo:tribuo-clustering-kmeans:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-regression-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-classification-sgd:4.2.1
          |    \--- compileClasspath
          +--- org.tribuo:tribuo-common-sgd:4.2.1
          |    +--- org.tribuo:tribuo-regression-sgd:4.2.1 (*)
          |    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)
          +--- org.tribuo:tribuo-anomaly-core:4.2.1
          |    \--- org.tribuo:tribuo-anomaly-libsvm:4.2.1
          |         \--- compileClasspath
          \--- org.tribuo:tribuo-common-tree:4.2.1
               \--- org.tribuo:tribuo-classification-core:4.2.1
                    \--- org.tribuo:tribuo-classification-sgd:4.2.1 (*)

Thanks, can you please show the same insight for plugin?

➜  ml-commons git:(backport_cve_fix) ./gradlew opensearch-ml-plugin:dependencyInsight --dependency commons-beanutils
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::load has been called by net.rubygrapefruit.platform.internal.NativeLibraryLoader in an unnamed module (file:/Users/pybot/.gradle/wrapper/dists/gradle-8.14-all/c2qonpi39x1mddn7hk5gh9iqj/gradle-8.14/lib/native-platform-0.22-milestone-28.jar)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.14
  OS Info               : Mac OS X 15.5 (aarch64)
  JDK Version           : 24 (Homebrew JDK)
  JAVA_HOME             : /opt/homebrew/Cellar/openjdk/24.0.2/libexec/openjdk.jdk/Contents/Home
  Random Testing Seed   : 595F4B25A5556B5E
  Crypto Standard       : any-supported
=======================================

> Task :opensearch-ml-plugin:dependencyInsight
No dependencies matching given input were found in configuration ':opensearch-ml-plugin:compileClasspath'

@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Error
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Error
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Error
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Error
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Failure
@dhrubo-os dhrubo-os had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:00 — with GitHub Actions Failure
@dhrubo-os
Copy link
Collaborator

@pyek-bot I took update from main and it's not showing option to approve the workflow. Can you please push an empty commit?

Signed-off-by: Pavan Yekbote <pybot@amazon.com>
@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

@pyek-bot I took update from main and it's not showing option to approve the workflow. Can you please push an empty commit?

Can you try now?

@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:03 — with GitHub Actions Failure
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 23:03 — with GitHub Actions Inactive
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:03 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:03 — with GitHub Actions Failure
@pyek-bot
Copy link
Collaborator Author

pyek-bot commented Aug 5, 2025

Windows CI Failed due to flaky test:

124 tests completed, 2 failed, 11 skipped
Tests with failures:
 - org.opensearch.ml.rest.RestMLRAGSearchProcessorIT.testBM25WithBedrockConverseUsingLlmMessages
 - org.opensearch.ml.rest.RestMLRAGSearchProcessorIT.testBM25WithBedrockConverse

@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval August 5, 2025 23:51 — with GitHub Actions Failure
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 23:51 — with GitHub Actions Inactive
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 23:51 — with GitHub Actions Inactive
@pyek-bot pyek-bot temporarily deployed to ml-commons-cicd-env-require-approval August 5, 2025 23:51 — with GitHub Actions Inactive
@codecov
Copy link

codecov bot commented Aug 6, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.93%. Comparing base (6cd0beb) to head (481e2a5).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #4062      +/-   ##
============================================
- Coverage     80.94%   80.93%   -0.02%     
+ Complexity     8259     8257       -2     
============================================
  Files           713      713              
  Lines         35963    35963              
  Branches       4042     4042              
============================================
- Hits          29111    29105       -6     
- Misses         5082     5083       +1     
- Partials       1770     1775       +5     
Flag Coverage Δ
ml-commons 80.93% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dhrubo-os dhrubo-os merged commit 4b4b409 into opensearch-project:main Aug 6, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants