Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
27 changes: 4 additions & 23 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,8 @@

*Description of changes:*

*CheckList:*
[ ] Commits are signed per the DCO using --signoff

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/index-management/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).
27 changes: 25 additions & 2 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,25 @@
## Code of Conduct
This project has adopted an [Open Source Code of Conduct](https://opendistro.github.io/for-elasticsearch/codeofconduct.html).

This code of conduct applies to all spaces provided by the OpenSource project including in code, documentation, issue trackers, mailing lists, chat channels, wikis, blogs, social media and any other communication channels used by the project.


**Our open source communities endeavor to:**

* Be Inclusive: We are committed to being a community where everyone can join and contribute. This means using inclusive and welcoming language.
* Be Welcoming: We are committed to maintaining a safe space for everyone to be able to contribute.
* Be Respectful: We are committed to encouraging differing viewpoints, accepting constructive criticism and work collaboratively towards decisions that help the project grow. Disrespectful and unacceptable behavior will not be tolerated.
* Be Collaborative: We are committed to supporting what is best for our community and users. When we build anything for the benefit of the project, we should document the work we do and communicate to others on how this affects their work.


**Our Responsibility. As contributors, members, or bystanders we each individually have the responsibility to behave professionally and respectfully at all times. Disrespectful and unacceptable behaviors include, but are not limited to:**

* The use of violent threats, abusive, discriminatory, or derogatory language;
* Offensive comments related to gender, gender identity and expression, sexual orientation, disability, mental illness, race, political or religious affiliation;
* Posting of sexually explicit or violent content;
* The use of sexualized language and unwelcome sexual attention or advances;
* Public or private harassment of any kind;
* Publishing private information, such as physical or electronic address, without permission;
* Other conduct which could reasonably be considered inappropriate in a professional setting;
* Advocating for or encouraging any of the above behaviors.
* Enforcement and Reporting Code of Conduct Issues:

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported. [Contact us](mailto:opensource-codeofconduct@amazon.com). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances.
55 changes: 50 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ information to effectively respond to your bug report or contribution.

We welcome you to use the GitHub issue tracker to report bugs or suggest features.

When filing an issue, please check [existing open](https://github.com/mauve-hedgehog/index-lifecycle-management/issues), or [recently closed](https://github.com/mauve-hedgehog/index-lifecycle-management/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
When filing an issue, please check [existing open](https://github.com/opensearch-project/index-management/issues), or [recently closed](https://github.com/opensearch-project/index-management/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:

* A reproducible test case or series of steps
Expand Down Expand Up @@ -45,17 +45,62 @@ Looking at the existing issues is a great way to find something to contribute on


## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
opensource-codeofconduct@amazon.com with any additional questions or comments.
See the [Code of Conduct](./CODE_OF_CONDUCT.md)


## Security issue notifications
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.


## Developer Certificate of Origin

OpenSearch is an open source product released under the Apache 2.0 license (see either [the Apache site](https://www.apache.org/licenses/LICENSE-2.0) or the [LICENSE file](./LICENSE)). The Apache 2.0 license allows you to freely use, modify, distribute, and sell your own products that include Apache 2.0 licensed software.

We respect intellectual property rights of others and we want to make sure all incoming contributions are correctly attributed and licensed. A Developer Certificate of Origin (DCO) is a lightweight mechanism to do that.

The DCO is a declaration attached to every contribution made by every developer. In the commit message of the contribution, the developer simply adds a `Signed-off-by` statement and thereby agrees to the DCO, which you can find below or at [DeveloperCertificate.org](http://developercertificate.org/).

```
Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the
best of my knowledge, is covered under an appropriate open
source license and I have the right under that license to
submit that work with modifications, whether created in whole
or in part by me, under the same open source license (unless
I am permitted to submit under a different license), as
Indicated in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including
all personal information I submit with it, including my
sign-off) is maintained indefinitely and may be redistributed
consistent with this project or the open source license(s)
involved.
```
We require that every contribution to OpenSearch is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms.

Each commit must include a DCO which looks like this

```
Signed-off-by: Jane Smith <jane.smith@email.com>
```
You may type this line on your own when writing your commit messages. However, if your user.name and user.email are set in your git configs, you can use `-s` or `– – signoff` to add the `Signed-off-by` line to the end of the commit message.



## Licensing

See the [LICENSE](https://github.com/mauve-hedgehog/index-lifecycle-management/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
See the [LICENSE](./LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.
8 changes: 8 additions & 0 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# OpenSearch Maintainers

## Maintainers
| Maintainer | GitHub ID | Affiliation |
| --------------- | --------- | ----------- |
| Charlotte | [CEHENKLE](https://github.com/CEHENKLE) | Amazon |
| Nick Knize | [nknize](https://github.com/nknize) | Amazon |
| Praveen Sameneni | [praveensameneni](https://github.com/praveensameneni) | Amazon |
14 changes: 12 additions & 2 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -1,2 +1,12 @@
Index Management
Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
OpenSearch
Copyright 2021 OpenSearch Contributors

This product includes software developed by
Elasticsearch (http://www.elastic.co).
Copyright 2009-2018 Elasticsearch

This product includes software developed by The Apache Software
Foundation (http://www.apache.org/).

This product includes software developed by
Joda.org (http://www.joda.org/).
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
![PRs welcome!](https://img.shields.io/badge/PRs-welcome!-success)


# Open Distro for Elasticsearch Index Management
# OpenSearch Index Management

Open Distro for Elasticsearch Index Management provides a suite of features to monitor and manage indexes.
OpenSearch Index Management provides a suite of features to monitor and manage indexes.

It currently contains an automated system for managing and optimizing indices throughout their life, Index State Management.

Expand Down Expand Up @@ -62,7 +62,7 @@ Please see our [documentation](https://opendistro.github.io/for-elasticsearch-do

The project in this package uses the [Gradle](https://docs.gradle.org/current/userguide/userguide.html) build system. Gradle comes with excellent documentation that should be your first stop when trying to figure out how to operate or modify the build.

However, to build the `index management` plugin project, we also use the Elastic build tools for Gradle. These tools are idiosyncratic and don't always follow the conventions and instructions for building regular Java code using Gradle. Not everything in `index management` will work the way it's described in the Gradle documentation. If you encounter such a situation, the Elastic build tools [source code](https://github.com/elastic/elasticsearch/tree/master/buildSrc/src/main/groovy/org/elasticsearch/gradle) is your best bet for figuring out what's going on.
However, to build the `index management` plugin project, we also use the OpenSearch build tools for Gradle. These tools are idiosyncratic and don't always follow the conventions and instructions for building regular Java code using Gradle. Not everything in `index management` will work the way it's described in the Gradle documentation. If you encounter such a situation, the OpenSearch build tools [source code](https://github.com/opensearch-project/OpenSearch/tree/main/buildSrc/src/main/groovy/org/opensearch/gradle) is your best bet for figuring out what's going on.

### Building from the command line

Expand All @@ -78,7 +78,7 @@ When launching a cluster using one of the above commands, logs are placed in `bu

### Debugging

Sometimes it is useful to attach a debugger to either the Elasticsearch cluster or the integ tests to see what's going on. When running unit tests, hit **Debug** from the IDE's gutter to debug the tests. For the Elasticsearch cluster or the integ tests, first, make sure start a debugger listening on port `5005`.
Sometimes it is useful to attach a debugger to either the OpenSearch cluster or the integ tests to see what's going on. When running unit tests, hit **Debug** from the IDE's gutter to debug the tests. For the OpenSearch cluster or the integ tests, first, make sure start a debugger listening on port `5005`.

To debug the server code, run:

Expand All @@ -92,7 +92,7 @@ OR
./gradlew run --debug-jvm # to just start a cluster that can be debugged
```

The Elasticsearch server JVM will connect to a debugger attached to `localhost:5005`.
The OpenSearch server JVM will connect to a debugger attached to `localhost:5005`.

The IDE needs to listen for the remote JVM. If using Intellij you must set your debug configuration to "Listen to remote JVM" and make sure "Auto Restart" is checked.
You must start your debugger to listen for remote JVM before running the commands.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
*/

/**
* ES Plugin build tools don't work with the Gradle Jacoco Plugin to report coverage out of the box.
* OpenSearch Plugin build tools don't work with the Gradle Jacoco Plugin to report coverage out of the box.
* https://github.com/elastic/elasticsearch/issues/28867.
*
* This code sets up coverage reporting manually for ES plugin tests. This is complicated because:
* 1. The ES integTest Task doesn't implement Gradle's JavaForkOptions so we have to manually start the jacoco agent with the test JVM
* This code sets up coverage reporting manually for OpenSearch plugin tests. This is complicated because:
* 1. The OpenSearch integTest Task doesn't implement Gradle's JavaForkOptions so we have to manually start the jacoco agent with the test JVM
* 2. The cluster nodes are stopped using 'kill -9' which means jacoco can't dump it's execution output to a file on VM shutdown
* 3. The Java Security Manager prevents JMX from writing execution output to the file.
*
Expand Down
4 changes: 2 additions & 2 deletions build-tools/pkgbuild.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ afterEvaluate {
release = isSnapshot ? "0.1" : '1'
version = "${project.version}" - "-SNAPSHOT"

into '/usr/share/elasticsearch/plugins'
into '/usr/share/opensearch/plugins'
from(zipTree(bundlePlugin.archivePath)) {
into esplugin.name
into opensearchplugin.name
}

user 'root'
Expand Down
48 changes: 30 additions & 18 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,32 @@
* permissions and limitations under the License.
*/

import org.elasticsearch.gradle.testclusters.TestClusterConfiguration
import org.opensearch.gradle.testclusters.TestClusterConfiguration

import java.util.concurrent.TimeUnit
import java.util.function.Predicate

buildscript {
ext {
es_version = System.getProperty("es.version", "7.10.2")
es_version = System.getProperty("es.version", "7.10.3")
kotlin_version = System.getProperty("kotlin.version", "1.3.72")
}

repositories {
mavenCentral()
maven { url "https://plugins.gradle.org/m2/" }
maven {
url = 's3://search-vemsarat/'
credentials(AwsCredentials) {
accessKey = System.env.AWS_ACCESS_KEY_ID ?: findProperty('aws_access_key_id')
secretKey = System.env.AWS_SECRET_ACCESS_KEY ?: findProperty('aws_secret_access_key')
}
}
// For local publish dependency
mavenLocal()
}

dependencies {
classpath "org.elasticsearch.gradle:build-tools:${es_version}"
classpath "org.opensearch.gradle:build-tools:${es_version}-SNAPSHOT"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${kotlin_version}"
classpath "org.jetbrains.kotlin:kotlin-allopen:${kotlin_version}"
classpath "io.gitlab.arturbosch.detekt:detekt-gradle-plugin:1.0.0-RC15"
Expand All @@ -48,9 +54,9 @@ plugins {
apply plugin: 'java'
apply plugin: 'jacoco'
apply plugin: 'idea'
apply plugin: 'elasticsearch.esplugin'
apply plugin: 'elasticsearch.testclusters'
apply plugin: 'elasticsearch.rest-test'
apply plugin: 'opensearch.opensearchplugin'
apply plugin: 'opensearch.testclusters'
apply plugin: 'opensearch.rest-test'
apply plugin: 'io.gitlab.arturbosch.detekt'
apply plugin: 'org.jetbrains.kotlin.jvm'
apply plugin: 'org.jetbrains.kotlin.plugin.allopen'
Expand All @@ -65,7 +71,6 @@ configurations.all {
force 'com.google.guava:guava:30.0-jre'
force 'com.puppycrawl.tools:checkstyle:8.29'
force 'commons-codec:commons-codec:1.13'
// force 'org.apache.logging.log4j:log4j-core:2.13.2'
force 'org.apache.httpcomponents:httpclient:4.5.13'
force 'org.apache.httpcomponents:httpclient-osgi:4.5.13'
force 'org.apache.httpcomponents.client5:httpclient5:5.0.3'
Expand All @@ -80,13 +85,13 @@ def usingRemoteCluster = System.properties.containsKey('tests.rest.cluster') ||
def usingMultiNode = project.properties.containsKey('numNodes')
// Only apply jacoco test coverage if we are running a local single node cluster
if (!usingRemoteCluster && !usingMultiNode) {
apply from: 'build-tools/esplugin-coverage.gradle'
apply from: 'build-tools/coverage.gradle'
}
check.dependsOn jacocoTestReport

esplugin {
name 'opendistro-index-management'
description 'Open Distro Index Management Plugin'
opensearchplugin {
name 'opensearch-index-management'
description 'OpenSearch Index Management Plugin'
classname 'com.amazon.opendistroforelasticsearch.indexmanagement.IndexManagementPlugin'
extendedPlugins = ['opendistro-job-scheduler']
}
Expand Down Expand Up @@ -124,17 +129,17 @@ group = "com.amazon.opendistroforelasticsearch"
version = "${opendistroVersion}.0"

dependencies {
compileOnly "org.elasticsearch:elasticsearch:${es_version}"
compileOnly "com.amazon.opendistroforelasticsearch:opendistro-job-scheduler-spi:1.13.0.0"
compileOnly "org.opensearch:opensearch:${es_version}-SNAPSHOT"
compileOnly "com.amazon.opendistroforelasticsearch:opendistro-job-scheduler-spi:1.15.0.0-SNAPSHOT"
compile "org.jetbrains.kotlin:kotlin-stdlib:${kotlin_version}"
compile "org.jetbrains.kotlin:kotlin-stdlib-common:${kotlin_version}"
compile 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.7'
compile "org.jetbrains:annotations:13.0"
compile "com.amazon.opendistroforelasticsearch:notification:1.13.1.0"
compile "com.amazon.opendistroforelasticsearch:common-utils:1.13.0.0"
compile "com.amazon.opendistroforelasticsearch:notification:1.15.0.0"
compile "com.amazon.opendistroforelasticsearch:common-utils:1.15.0.0"
compile "com.github.seancfoley:ipaddress:5.3.3"

testCompile "org.elasticsearch.test:framework:${es_version}"
testCompile "org.opensearch.test:framework:${es_version}-SNAPSHOT"
testCompile "org.jetbrains.kotlin:kotlin-test:${kotlin_version}"
testImplementation "com.nhaarman.mockitokotlin2:mockito-kotlin:2.2.0"
testCompile "org.mockito:mockito-core:2.23.0"
Expand All @@ -143,6 +148,13 @@ dependencies {
}

repositories {
maven {
url = 's3://search-vemsarat/'
credentials(AwsCredentials) {
accessKey = System.env.AWS_ACCESS_KEY_ID ?: findProperty('aws_access_key_id')
secretKey = System.env.AWS_SECRET_ACCESS_KEY ?: findProperty('aws_secret_access_key')
}
}
// TODO: remove mavenLocal once notification is published to maven
mavenLocal()
}
Expand Down Expand Up @@ -319,7 +331,7 @@ apply from: 'build-tools/pkgbuild.gradle'
// This IT is to simulate the situation
// when there are old version (without metadata change)
// and new version mixed in one cluster
import org.elasticsearch.gradle.test.RestIntegTestTask
import org.opensearch.gradle.test.RestIntegTestTask

def mixedClusterTest = project.tasks.create('mixedCluster', RestIntegTestTask.class)
def mixedClusterFlag = findProperty('mixed') as Boolean ?: false
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@
# permissions and limitations under the License.
#

version = 1.13.2
version = 1.15.0
Loading