Skip to content

Commit

Permalink
Merge branch 'main' into patch-1
Browse files Browse the repository at this point in the history
Signed-off-by: Thomas Farr <tsfarr@amazon.com>
  • Loading branch information
Xtansia authored Sep 30, 2024
2 parents e60923d + 18a8460 commit 0c62ae9
Show file tree
Hide file tree
Showing 262 changed files with 23,317 additions and 4,957 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/release-drafter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
with:
secret: ${{ github.TOKEN }}
approvers: ${{ steps.get_data.outputs.approvers }}
minimum-approvals: 2
minimum-approvals: 1
issue-title: 'Release opensearch-java : ${{ steps.get_data.outputs.version }}'
issue-body: "Please approve or deny the release of opensearch-java. **VERSION**: ${{ steps.get_data.outputs.version }} **TAG**: ${{ github.ref_name }} **COMMIT**: ${{ github.sha }}"
exclude-workflow-initiator-as-approver: true
Expand Down
20 changes: 17 additions & 3 deletions .github/workflows/test-integration-unreleased.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,12 @@ jobs:
matrix:
entry:
- { opensearch_ref: '1.x', java: 11 }
- { opensearch_ref: '2.x', java: 11 }
# See please https://docs.gradle.org/8.10/userguide/upgrading_version_8.html#minimum_daemon_jvm_version
- { opensearch_ref: '2.x', java: 11, gradle-runtime: 17 }
- { opensearch_ref: '2.x', java: 17 }
- { opensearch_ref: '2.x', java: 21 }
- { opensearch_ref: 'main', java: 11 }
# See please https://docs.gradle.org/8.10/userguide/upgrading_version_8.html#minimum_daemon_jvm_version
- { opensearch_ref: 'main', java: 11, gradle-runtime: 17 }
- { opensearch_ref: 'main', java: 17 }
- { opensearch_ref: 'main', java: 21 }
steps:
Expand Down Expand Up @@ -48,10 +50,22 @@ jobs:
path: opensearch/distribution/archives/linux-tar/build/distributions
key: ${{ steps.get-key.outputs.key }}

- name: Set up Gradle JDK runtime
if: matrix.entry.gradle-runtime
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.entry.gradle-runtime }}
distribution: temurin

- name: Set JAVA${{ matrix.entry.java }}_HOME
shell: bash
run: |
echo "JAVA${{ matrix.entry.java }}_HOME=$JAVA_HOME_${{ matrix.entry.java }}_${{ runner.arch }}" >> $GITHUB_ENV
- name: Assemble OpenSearch
if: steps.cache-restore.outputs.cache-hit != 'true'
working-directory: opensearch
run: ./gradlew :distribution:archives:linux-tar:assemble
run: ./gradlew :distribution:archives:linux-tar:assemble -Druntime.java=${{ matrix.java }}

- name: Save cached build
if: steps.cache-restore.outputs.cache-hit != 'true'
Expand Down
47 changes: 42 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,15 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

## [Unreleased 3.0]
### Dependencies
- Bumps `org.junit:junit-bom` from 5.10.2 to 5.10.3
- Bumps `org.junit:junit-bom` from 5.10.2 to 5.11.1
- Bumps `org.owasp.dependencycheck` from 10.0.2 to 10.0.3
- Bumps `org.eclipse.parsson:parsson` from 1.1.6 to 1.1.7
- Bumps `org.hamcrest:hamcrest` from 2.2 to 3.0
- Bumps `com.github.jk1.dependency-license-report` from 2.8 to 2.9
- Bumps `org.apache.httpcomponents.core5:httpcore5-h2` from 5.2.5 to 5.3
- Bumps `org.apache.httpcomponents.core5:httpcore5` from 5.2.5 to 5.3
- Bumps `org.apache.httpcomponents.client5:httpclient5` from 5.3.1 to 5.4
- Bumps `io.github.classgraph:classgraph` from 4.8.175 to 4.8.176

This section is for maintaining a changelog for all breaking changes for the client that cannot be released in the 2.x line. All other non-breaking changes should be added to [Unreleased 2.x] section.

Expand All @@ -18,6 +24,8 @@ This section is for maintaining a changelog for all breaking changes for the cli
- Migrate client transports to Apache HttpClient / Core 5.x ([#246](https://github.com/opensearch-project/opensearch-java/pull/246))
- Changed SearchAfter of SearchRequest type to FieldValue instead of String ([#769](https://github.com/opensearch-project/opensearch-java/pull/769))
- Changed type of `DanglingIndex`'s `creationDateMillis` field from `String` to `long` ([#1124](https://github.com/opensearch-project/opensearch-java/pull/1124))
- Changed type of `ShardStatistics`'s `total`, `successful`, `failed` & `skipped` fields from `Number` to `int/Integer` ([#1158](https://github.com/opensearch-project/opensearch-java/pull/1158))
- Unified `tasks.Info` and `tasks.State` classes into `tasks.TaskInfo` ([#1187](https://github.com/opensearch-project/opensearch-java/pull/1187))

### Deprecated
- Deprecate RestClientTransport ([#536](https://github.com/opensearch-project/opensearch-java/pull/536))
Expand All @@ -26,29 +34,56 @@ This section is for maintaining a changelog for all breaking changes for the cli

### Fixed
- Fix version and build ([#254](https://github.com/opensearch-project/opensearch-java/pull/254))
- Fixed deserializing `tasks.ListResponse` when using `GroupBy.None` ([#1187](https://github.com/opensearch-project/opensearch-java/pull/1187))

### Security

## [Unreleased 2.x]

### Added
- Add `hashCode` and `equals` implementations ([#312](https://github.com/opensearch-project/opensearch-java/pull/312)).

### Dependencies
- Bumps `org.junit:junit-bom` from 5.10.3 to 5.11.0

### Changed
- Changed URL path encoding to own implementation adapted from Apache HTTP Client 5's ([#1109](https://github.com/opensearch-project/opensearch-java/pull/1109))

### Deprecated

### Removed

### Fixed
- Fixed error when deserializing a normalizer without 'type' ([#1111](https://github.com/opensearch-project/opensearch-java/pull/1111))
- Fixed deserialization of SearchRequest when `_source` is an array ([#1117](https://github.com/opensearch-project/opensearch-java/pull/1117))
- Fix queries not preserving boost and name when converted to builders ([#1181](https://github.com/opensearch-project/opensearch-java/pull/1181))
- Remove required check on ShardStats.shards ([#1177](https://github.com/opensearch-project/opensearch-java/pull/1177))
- Fixed `TextEmbeddingProcessor` returning the wrong processor kind ([#1127](https://github.com/opensearch-project/opensearch-java/pull/1127))

### Security

## [2.14.0] - 08/27/2024
### Added
- Added `queryImage` (query_image) field to `NeuralQuery`, following definition in ([Neural Query](https://opensearch.org/docs/latest/query-dsl/specialized/neural/)) ([#1137](https://github.com/opensearch-project/opensearch-java/pull/1138))
- Added `cancelAfterTimeInterval` to `SearchRequest` and `MsearchRequest` ([#1147](https://github.com/opensearch-project/opensearch-java/pull/1147))
- Added the `ml` namespace operations ([#1158](https://github.com/opensearch-project/opensearch-java/pull/1158))
- Added `IndexTemplateMapping.Builder#withJson`, `SourceField.Builder#withJson` and `IndexSettings.Builder#withJson` for streamlining deserialization ([#1148](https://github.com/opensearch-project/opensearch-java/pull/1148))

### Dependencies
- Bumps `commons-logging:commons-logging` from 1.3.3 to 1.3.4
- Bumps `io.github.classgraph:classgraph` from 4.8.174 to 4.8.175

### Fixed
- Fixed handling of NodeInfo fields that are not returned by Amazon OpenSearch Service ([#1132](https://github.com/opensearch-project/opensearch-java/pull/1132))

## [2.13.0] - 08/01/2024
### Added
- Added `searchAfterVals` to `SearchRequest` to allow passing arbitrary `FieldValue`s to `search_after` ([#1105](https://github.com/opensearch-project/opensearch-java/pull/1105))

### Changed
- Changed URL path encoding to own implementation adapted from Apache HTTP Client 5's ([#1109](https://github.com/opensearch-project/opensearch-java/pull/1109))

### Fixed
- Fixed error when deserializing a normalizer without 'type' ([#1111](https://github.com/opensearch-project/opensearch-java/pull/1111))
- Fixed deserialization of SearchRequest when `_source` is an array ([#1117](https://github.com/opensearch-project/opensearch-java/pull/1117))

## [2.12.0] - 07/22/2024
### Added
- Added support for the `parent` aggregation type ([#706](https://github.com/opensearch-project/opensearch-java/pull/706))
Expand Down Expand Up @@ -497,7 +532,9 @@ This section is for maintaining a changelog for all breaking changes for the cli
### Security

[Unreleased 3.0]: https://github.com/opensearch-project/opensearch-java/compare/2.x...HEAD
[Unreleased 2.x]: https://github.com/opensearch-project/opensearch-java/compare/v2.12.0...2.x
[Unreleased 2.x]: https://github.com/opensearch-project/opensearch-java/compare/v2.14.0...2.x
[2.14.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.13.0...v2.14.0
[2.13.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.12.0...v2.13.0
[2.12.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.11.1...v2.12.0
[2.11.1]: https://github.com/opensearch-project/opensearch-java/compare/v2.11.0...v2.11.1
[2.11.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.10.4...v2.11.0
Expand Down
30 changes: 29 additions & 1 deletion UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,32 @@ After:
```

### DanglingIndex creationDateMillis type
- The type of `DanglingIndex`'s `creationDateMillis` field has been corrected from a `String` to a `long`.
- The type of `DanglingIndex`'s `creationDateMillis` field has been corrected from a `String` to a `long`.

### ShardStatistics properties types
- The type of the `total`, `successful` and `failed` fields has been corrected from `Number` to `int`.
- The type of the `skipped` field has been corrected from `Number` to `Integer`.

### Unified tasks.Info & tasks.State classes into tasks.TaskInfo
- The `tasks.Info` and `tasks.State` classes have been unified into `tasks.TaskInfo`, this affects:
- `TaskExecutingNode`'s `tasks` field.
- `GetTasksResponse`'s `task` field.
- `core.update_by_query_rethrottle.UpdateByQueryRethrottleNode`'s `tasks` field.
- The `headers` field is now a `Map<String, String>` instead of a `Map<String, List<String>>`.

### tasks.ListResponse properties lifted to tasks.TaskListResponseBase
- All fields previously defined on `tasks.ListResponse` have been lifted to `tasks.TaskListResponseBase`.
- `DeleteByQueryRethrottleResponse` now extends `tasks.TaskListResponseBase` instead of `tasks.ListResponse`.
- The `tasks` field is now a `TaskInfos` union type instead of a `Map<String, Info>` to correctly handle `groupBy` parents or none.

### GetTasksResponse response type
- The type of `GetTasksResponse`'s `response` field has been changed from `tasks.Status` to `tasks.TaskResponse`.

### VerifyRepositoryRequest property naming
- The `name` field, getter and builder method have been renamed to `repository`.

### CleanupRepositoryRequest property naming
- The `name` field, getter and builder method have been renamed to `repository`.

### CloneSnapshotRequest timeout removal
- The `timeout` field, getter and builder method have been removed from `CloneSnapshotRequest` as it is not supported by OpenSearch.
2 changes: 1 addition & 1 deletion build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ allprojects {
mavenLocal()
maven(url = "https://aws.oss.sonatype.org/content/repositories/snapshots")
mavenCentral()
maven(url = "https://plugins.gradle.org/m2/")
gradlePluginPortal()
}
}

Expand Down
12 changes: 9 additions & 3 deletions buildSrc/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,19 @@
* GitHub history for details.
*/

dependencies {
implementation("org.ajoberstar.grgit:grgit-gradle:5.2.2")
plugins {
`kotlin-dsl`
}

repositories {
mavenLocal()
maven(url = "https://aws.oss.sonatype.org/content/repositories/snapshots")
mavenCentral()
maven(url = "https://plugins.gradle.org/m2/")
gradlePluginPortal()
}

dependencies {
implementation("org.ajoberstar.grgit:grgit-gradle:5.2.2")
implementation("com.diffplug.spotless", "spotless-plugin-gradle", "6.25.0")
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
plugins {
id("com.diffplug.spotless")
}

interface SpotlessConventionsPluginExtension {
val eclipseFormatterConfigFile: RegularFileProperty
}

val extension = project.extensions.create<SpotlessConventionsPluginExtension>("spotlessConventions")

extension.eclipseFormatterConfigFile.convention(rootProject.layout.projectDirectory.file("buildSrc/formatterConfig.xml"))

spotless {
java {
target("**/*.java")

licenseHeaderFile(rootProject.file("LICENSE_HEADER.txt"))
.named("PrimaryLicenseHeader")
.onlyIfContentMatches("^((?!Licensed to Elasticsearch)[\\s\\S])*$")
.delimiter("(package |//-----)")

licenseHeaderFile(rootProject.file("LICENSE_HEADER_FORKED.txt"))
.named("ForkedLicenseHeader")
.onlyIfContentMatches("Licensed to Elasticsearch")
.delimiter("(package |//-----)")

// Use the default importOrder configuration
importOrder()
removeUnusedImports()

eclipse().configFile(extension.eclipseFormatterConfigFile)

trimTrailingWhitespace()
endWithNewline()

// NOTE: Any time a custom step below is modified, bump this number.
// Allows up-to-date checks to work correctly with custom steps.
bumpThisNumberIfACustomStepChanges(1)

val wildcardImportRegex = Regex("""^import\s+(?:static\s+)?[^*\s]+\.\*;$""", RegexOption.MULTILINE)
custom("Refuse wildcard imports") { contents ->
// Wildcard imports can't be resolved by spotless itself.
// This will require the developer themselves to adhere to best practices.
val wildcardImports = wildcardImportRegex.findAll(contents)
if (wildcardImports.any()) {
var msg = """
Please replace the following wildcard imports with explicit imports ('spotlessApply' cannot resolve this issue):
""".trimIndent()
wildcardImports.forEach {
msg += "\n\t- ${it.value}"
}
msg += "\n"
throw AssertionError(msg)
}
contents
}
}
}
Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
4 changes: 2 additions & 2 deletions gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.9-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.1-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionSha256Sum=d725d707bfabd4dfdc958c624003b3c80accc03f7037b5122c4b1d0ef15cecab
distributionSha256Sum=1541fa36599e12857140465f3c91a97409b4512501c26f9631fb113e392c5bd1
43 changes: 43 additions & 0 deletions guides/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
- [Serialization](#serialization)
- [Using toJsonString](#using-tojsonstring)
- [Manual Serialization](#manual-serialization)
- [Deserialization](#deserialization)
- [Using withJson](#using-withjson)
- [Using static _DESERIALIZER](#using-static-_deserializer)


# Working With JSON
Expand Down Expand Up @@ -50,4 +53,44 @@ private String toJson(JsonpSerializable object) {
throw new UncheckedIOException(ex);
}
}
```

## Deserialization

For demonstration let's consider an IndexTemplateMapping JSON String.

```java

String stringTemplate =
"{\"mappings\":{\"properties\":{\"age\":{\"type\":\"integer\"}}},\"settings\":{\"number_of_shards\":\"2\",\"number_of_replicas\":\"1\"}}";
```
### Using withJson
For classes, Builders of which implements `PlainDeserializable` interface, a default `withJson` method is provided.
The withJson method returns the Builder enabling you to chain Builder methods for additional configuration.
This implementation uses `jakarta.json.spi.JsonProvider` SPI to discover the available JSON provider instance
from the classpath and to create a new mapper. The `JsonpUtils` utility class streamlines this deserialization process.
The following code example demonstrates how to use the `withJson` method to deserialize objects:

```java
InputStream inputStream = new ByteArrayInputStream(stringTemplate.getBytes(StandardCharsets.UTF_8));
IndexTemplateMapping indexTemplateMapping = new IndexTemplateMapping.Builder().withJson(inputStream).build();
```


### Using static _DESERIALIZER
For classes annotated with `@JsonpDeserializable`, a static field _DESERIALIZER is provided,
which takes a mapper and a parser as arguments and returns the instance of the json value passed in the parser.
Notice that this way you cannot further customize the instance, the state of which will solely depend on the json value parsed.

The following sample code demonstrates how to serialize an instance of a Java class:

```java
private IndexTemplateMapping getInstance(String templateJsonString) {
InputStream inputStream = new ByteArrayInputStream(templateJsonString.getBytes(StandardCharsets.UTF_8));
JsonbJsonpMapper mapper = new JsonbJsonpMapper();
try (JsonParser parser = mapper.jsonProvider().createParser(inputStream)) {
IndexTemplateMapping indexTemplateMapping = new IndexTemplateMapping._DESERIALIZER.deserialize(parser, mapper);
return indexTemplateMapping;
}
}
```
Loading

0 comments on commit 0c62ae9

Please sign in to comment.