Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into main-throttling-bugfix
Browse files Browse the repository at this point in the history
Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>
  • Loading branch information
dhwanilpatel committed Dec 15, 2022
2 parents 5272511 + cb26035 commit 5a68a0e
Show file tree
Hide file tree
Showing 145 changed files with 1,102 additions and 651 deletions.
1 change: 1 addition & 0 deletions .ci/bwcVersions
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ BWC_VERSION:
- "2.3.1"
- "2.4.0"
- "2.4.1"
- "2.4.2"
- "2.5.0"
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Added jackson dependency to server ([#5366] (https://github.com/opensearch-project/OpenSearch/pull/5366))
- Added experimental extensions to main ([#5347](https://github.com/opensearch-project/OpenSearch/pull/5347))
- Adding support to register settings dynamically ([#5495](https://github.com/opensearch-project/OpenSearch/pull/5495))
- Add CI bundle pattern to distribution download ([#5348](https://github.com/opensearch-project/OpenSearch/pull/5348))


### Dependencies
Expand Down Expand Up @@ -52,6 +53,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Bumps `protobuf-java` from 3.21.7 to 3.21.9 ([#5319](https://github.com/opensearch-project/OpenSearch/pull/5319))
- Update Apache Lucene to 9.5.0-snapshot-a4ef70f ([#4979](https://github.com/opensearch-project/OpenSearch/pull/4979))
- Update to Gradle 7.6 and JDK-19 ([#4973](https://github.com/opensearch-project/OpenSearch/pull/4973))
- Bumps `protobuf-java` from 3.21.9 to 3.21.11 in /plugins/repository-hdfs ([#5519](https://github.com/opensearch-project/OpenSearch/pull/5519))
- Update Apache Lucene to 9.5.0-snapshot-d5cef1c ([#5570](https://github.com/opensearch-project/OpenSearch/pull/5570))

### Changed
- [CCR] Add getHistoryOperationsFromTranslog method to fetch the history snapshot from translogs ([#3948](https://github.com/opensearch-project/OpenSearch/pull/3948))
Expand All @@ -61,6 +64,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Support remote translog transfer for request level durability([#4480](https://github.com/opensearch-project/OpenSearch/pull/4480))
- Changed http code on create index API with bad input raising NotXContentException from 500 to 400 ([#4773](https://github.com/opensearch-project/OpenSearch/pull/4773))
- Change http code for DecommissioningFailedException from 500 to 400 ([#5283](https://github.com/opensearch-project/OpenSearch/pull/5283))
- Pre conditions check before updating weighted routing metadata ([#4955](https://github.com/opensearch-project/OpenSearch/pull/4955))

### Deprecated

Expand Down Expand Up @@ -101,6 +105,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Bump gradle-extra-configurations-plugin from 7.0.0 to 8.0.0 ([#4808](https://github.com/opensearch-project/OpenSearch/pull/4808))
### Changed
### Deprecated
- Refactor fuzziness interface on query builders ([#5433](https://github.com/opensearch-project/OpenSearch/pull/5433))

### Removed
### Fixed
- Fix 1.x compatibility bug with stored Tasks ([#5412](https://github.com/opensearch-project/OpenSearch/pull/5412))
Expand Down
4 changes: 4 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,10 @@ Use -Dtest.class and -Dtests.method to run a specific bwcTest test. For example
-Dtests.class=org.opensearch.upgrades.RecoveryIT \
-Dtests.method=testHistoryUUIDIsGenerated

Use `-PcustomDistributionDownloadType=bundle` to run the bwcTest against the test cluster with latest CI distribution bundle set up for the specified version; this property is default to min and exclusive choices between `bundle` and `min`:

./gradlew bwcTest -PcustomDistributionDownloadType=bundle

When running `./gradlew check`, minimal bwc checks are also run against compatible versions that are not yet released.

## BWC Testing against a specific remote/branch
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ public class DistributionDownloadPlugin implements Plugin<Project> {
private static final String RELEASE_PATTERN_LAYOUT = "/core/opensearch/[revision]/[module]-min-[revision](-[classifier]).[ext]";
private static final String SNAPSHOT_PATTERN_LAYOUT =
"/snapshots/core/opensearch/[revision]/[module]-min-[revision](-[classifier])-latest.[ext]";
private static final String BUNDLE_PATTERN_LAYOUT =
"/ci/dbc/distribution-build-opensearch/[revision]/latest/linux/x64/tar/dist/opensearch/[module]-[revision](-[classifier]).[ext]";

private NamedDomainObjectContainer<OpenSearchDistribution> distributionsContainer;
private NamedDomainObjectContainer<DistributionResolution> distributionsResolutionStrategiesContainer;
Expand Down Expand Up @@ -174,20 +176,39 @@ private static void setupDownloadServiceRepo(Project project) {
return;
}
Object customDistributionUrl = project.findProperty("customDistributionUrl");
// checks if custom Distribution Url has been passed by user from plugins
Object customDistributionDownloadType = project.findProperty("customDistributionDownloadType");
// distributionDownloadType is default min if is not specified; download the distribution from CI if is bundle
String distributionDownloadType = customDistributionDownloadType != null
&& customDistributionDownloadType.toString().equals("bundle") ? "bundle" : "min";
if (customDistributionUrl != null) {
addIvyRepo(project, DOWNLOAD_REPO_NAME, customDistributionUrl.toString(), FAKE_IVY_GROUP, "");
addIvyRepo(project, SNAPSHOT_REPO_NAME, customDistributionUrl.toString(), FAKE_SNAPSHOT_IVY_GROUP, "");
} else {
addIvyRepo(
project,
DOWNLOAD_REPO_NAME,
"https://artifacts.opensearch.org",
FAKE_IVY_GROUP,
"/releases" + RELEASE_PATTERN_LAYOUT,
"/release-candidates" + RELEASE_PATTERN_LAYOUT
);
addIvyRepo(project, SNAPSHOT_REPO_NAME, "https://artifacts.opensearch.org", FAKE_SNAPSHOT_IVY_GROUP, SNAPSHOT_PATTERN_LAYOUT);
return;
}
switch (distributionDownloadType) {
case "bundle":
addIvyRepo(project, DOWNLOAD_REPO_NAME, "https://ci.opensearch.org", FAKE_IVY_GROUP, BUNDLE_PATTERN_LAYOUT);
addIvyRepo(project, SNAPSHOT_REPO_NAME, "https://ci.opensearch.org", FAKE_SNAPSHOT_IVY_GROUP, BUNDLE_PATTERN_LAYOUT);
break;
case "min":
addIvyRepo(
project,
DOWNLOAD_REPO_NAME,
"https://artifacts.opensearch.org",
FAKE_IVY_GROUP,
"/releases" + RELEASE_PATTERN_LAYOUT,
"/release-candidates" + RELEASE_PATTERN_LAYOUT
);
addIvyRepo(
project,
SNAPSHOT_REPO_NAME,
"https://artifacts.opensearch.org",
FAKE_SNAPSHOT_IVY_GROUP,
SNAPSHOT_PATTERN_LAYOUT
);
break;
default:
throw new IllegalArgumentException("Unsupported property argument: " + distributionDownloadType);
}
}

Expand Down
4 changes: 2 additions & 2 deletions buildSrc/version.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
opensearch = 3.0.0
lucene = 9.5.0-snapshot-a4ef70f
lucene = 9.5.0-snapshot-d5cef1c

bundled_jdk_vendor = adoptium
bundled_jdk = 19.0.1+10
Expand All @@ -24,7 +24,7 @@ kotlin = 1.7.10
# when updating the JNA version, also update the version in buildSrc/build.gradle
jna = 5.5.0

netty = 4.1.84.Final
netty = 4.1.86.Final
joda = 2.10.13

# client dependencies
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
afe877ebf5ec4be7d17636b695015c449a523a3b
3 changes: 2 additions & 1 deletion modules/transport-netty4/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,8 @@ thirdPartyAudit {
'com.aayushatharva.brotli4j.Brotli4jLoader',
'com.aayushatharva.brotli4j.decoder.DecoderJNI$Status',
'com.aayushatharva.brotli4j.decoder.DecoderJNI$Wrapper',
'com.aayushatharva.brotli4j.encoder.Encoders',
'com.aayushatharva.brotli4j.encoder.BrotliEncoderChannel',
'com.aayushatharva.brotli4j.encoder.Encoder',
'com.aayushatharva.brotli4j.encoder.Encoder$Mode',
'com.aayushatharva.brotli4j.encoder.Encoder$Parameters',
// classes are missing
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
a66fa0ed2687eb33a2e53a17a6df61bfe3b3f2bd

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ee126da926ea202da3b21eb737788ef83b1db772

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
23674593f004959ae002ec348626eecf677191ae

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ac0ca067e4118533ad1038776fcd9d5f3058b7d4

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1dceab4662a9cc93faf87b237bb41103b1bc7f0e

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0bcb65230218286e6456b5d085cb42e67776eb70

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bad83d479f7bd8ea84eefd77c316435be4c97270

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7c7739c41fd110c3576e9faace332ee957f27203

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5e0e7fc1c337485cabcf7971faefe692b76f93a2

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
670d8f48ea9cba542e263d3ec6c3e2a33accc561

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
b1f42bad26470c8ef88096e0c8564a74223c52c9

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
f3cf74fa91da5133667f8916f93071fed231f2ee

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e633c2d0cd677e4f1cef5aadc6bdc65e8e898d98

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c12a2943e6f4977f15d489ac3e9802c5dfb3c4cc

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5d257928a34e586a7de9fc7d4a013868f7a1db74

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3d88f80ad07421b9470cb44a6f5b67dd47047b13

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c8de479f36a8457541fcbb0016c851bde3e67693

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ac0ca067e4118533ad1038776fcd9d5f3058b7d4

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
854264e7ad75887bc25b82eb38e4ee65c8b44dc3

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
2515d76be9671cc248bab77352edddd16bfa9436

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
a1e2ef79e4944b5d38092328c36c68e677a4b5f3

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5e0e7fc1c337485cabcf7971faefe692b76f93a2
2 changes: 1 addition & 1 deletion plugins/repository-hdfs/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ dependencies {
api 'org.apache.avro:avro:1.11.1'
api 'com.google.code.gson:gson:2.10'
runtimeOnly 'com.google.guava:guava:31.1-jre'
api 'com.google.protobuf:protobuf-java:3.21.9'
api 'com.google.protobuf:protobuf-java:3.21.11'
api "commons-logging:commons-logging:${versions.commonslogging}"
api 'commons-cli:commons-cli:1.5.0'
api "commons-codec:commons-codec:${versions.commonscodec}"
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
a6395c3d2f8699e8dc4fd1e38171f82045f4af7b
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c94f1937debcacbbeff48208bc2f7279088cbcdc

This file was deleted.

3 changes: 2 additions & 1 deletion plugins/transport-nio/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ thirdPartyAudit {
'com.aayushatharva.brotli4j.Brotli4jLoader',
'com.aayushatharva.brotli4j.decoder.DecoderJNI$Status',
'com.aayushatharva.brotli4j.decoder.DecoderJNI$Wrapper',
'com.aayushatharva.brotli4j.encoder.Encoders',
'com.aayushatharva.brotli4j.encoder.BrotliEncoderChannel',
'com.aayushatharva.brotli4j.encoder.Encoder',
'com.aayushatharva.brotli4j.encoder.Encoder$Mode',
'com.aayushatharva.brotli4j.encoder.Encoder$Parameters',

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
a66fa0ed2687eb33a2e53a17a6df61bfe3b3f2bd

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ee126da926ea202da3b21eb737788ef83b1db772

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
23674593f004959ae002ec348626eecf677191ae

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1dceab4662a9cc93faf87b237bb41103b1bc7f0e

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0bcb65230218286e6456b5d085cb42e67776eb70

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bad83d479f7bd8ea84eefd77c316435be4c97270

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7c7739c41fd110c3576e9faace332ee957f27203
3 changes: 2 additions & 1 deletion release-notes/opensearch.release-notes-2.4.1.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## 2022-12-07 Version 2.4.1 Release Notes
## 2022-12-13 Version 2.4.1 Release Notes

### Bug Fixes
* Fix 1.x compatibility bug with stored Tasks ([#5412](https://github.com/opensearch-project/opensearch/pull/5412)) ([#5440](https://github.com/opensearch-project/opensearch/pull/5440))
Expand All @@ -20,3 +20,4 @@
* Update Apache Lucene to 9.4.2 ([#5354](https://github.com/opensearch-project/opensearch/pull/5354)) ([#5361](https://github.com/opensearch-project/opensearch/pull/5361))
* Update Jackson to 2.14.1 ([#5346](https://github.com/opensearch-project/opensearch/pull/5346)) ([#5358](https://github.com/opensearch-project/opensearch/pull/5358))
* Bump nebula-publishing-plugin from v4.4.0 to v4.6.0. ([#5127](https://github.com/opensearch-project/opensearch/pull/5127)) ([#5131](https://github.com/opensearch-project/opensearch/pull/5131))
* Bump commons-compress from 1.21 to 1.22. ([#5520](https://github.com/opensearch-project/OpenSearch/pull/5520)) ([#5522](https://github.com/opensearch-project/opensearch/pull/5522))

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
f2440fe126dad78e95f901c0f7a6eeb66da09938

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
51677b84f823e352ab366f6a6bf87de8816650c4

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
b11b5c54ab26152c0db003c7a514f4e6c6825fdd

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
58bd60d4c3ec753eef0d904601ab6d726633a8db

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
97f11221b89e37c575ed9538d88b1529872abd80

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
84980688c8eb9fbdb597a4291713ee630653392d

This file was deleted.

Loading

0 comments on commit 5a68a0e

Please sign in to comment.