Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added point in time APIs #348

Merged
merged 1 commit into from
Jan 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Added jsdoc for documentation generation ([#335](https://github.com/opensearch-project/opensearch-js/issues/335))
- Documented Transport#request ([#335](https://github.com/opensearch-project/opensearch-js/issues/335))
- Documented all API methods ([#335](https://github.com/opensearch-project/opensearch-js/issues/335))
- Added point in time APIs ([#348](https://github.com/opensearch-project/opensearch-js/pull/348))
### Dependencies
- Bumps `xmlbuilder2` from 2.4.1 to 3.0.2
- Bumps `minimatch` from 3.0.4 to 3.1.2
Expand Down
173 changes: 112 additions & 61 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
- [Delete the index](#delete-the-index)

## Initializing a Client

```javascript
'use strict';

Expand Down Expand Up @@ -70,14 +71,14 @@ const client = new Client({
});
}),
}),
node: "https://search-xxx.region.es.amazonaws.com", // OpenSearch domain URL
node: 'https://search-xxx.region.es.amazonaws.com', // OpenSearch domain URL
});
```

#### Using AWS V3 SDK

```javascript
const { defaultProvider } = require("@aws-sdk/credential-provider-node"); // V3 SDK.
const { defaultProvider } = require('@aws-sdk/credential-provider-node'); // V3 SDK.
const { Client } = require('@opensearch-project/opensearch');
const { AwsSigv4Signer } = require('@opensearch-project/opensearch/aws');

Expand All @@ -97,101 +98,151 @@ const client = new Client({
return credentialsProvider();
},
}),
node: "https://search-xxx.region.es.amazonaws.com", // OpenSearch domain URL
node: 'https://search-xxx.region.es.amazonaws.com', // OpenSearch domain URL
});
```

## Create an Index

```javascript
console.log('Creating index:');

var index_name = 'books';
var settings = {
settings: {
index: {
number_of_shards: 4,
number_of_replicas: 3,
},
console.log('Creating index:');

var index_name = 'books';
var settings = {
settings: {
index: {
number_of_shards: 4,
number_of_replicas: 3,
},
};
},
};

var response = await client.indices.create({
index: index_name,
body: settings,
});
var response = await client.indices.create({
index: index_name,
body: settings,
});

console.log(response.body);
console.log(response.body);
```

## Add a Document to the Index

```javascript
console.log('Adding document:');

var document = {
title: 'The Outsider',
author: 'Stephen King',
year: '2018',
genre: 'Crime fiction',
};

var id = '1';

var response = await client.index({
id: id,
index: index_name,
body: document,
refresh: true,
});
console.log('Adding document:');

var document = {
title: 'The Outsider',
author: 'Stephen King',
year: '2018',
genre: 'Crime fiction',
};

var id = '1';

var response = await client.index({
id: id,
index: index_name,
body: document,
refresh: true,
});

console.log(response.body);
console.log(response.body);
```

## Search for the Document

```javascript
console.log('Search results:');

var query = {
query: {
match: {
title: {
query: 'The Outsider',
},
console.log('Search results:');

var query = {
query: {
match: {
title: {
query: 'The Outsider',
},
},
};
},
};

var response = await client.search({
index: index_name,
body: query,
});
var response = await client.search({
index: index_name,
body: query,
});

console.log(response.body.hits);
console.log(response.body.hits);
```

## Delete the document

```javascript
console.log('Deleting document:');
console.log('Deleting document:');

var response = await client.delete({
index: index_name,
id: id,
});
var response = await client.delete({
index: index_name,
id: id,
});

console.log(response.body);
console.log(response.body);
```

## Delete the index

```javascript
console.log('Deleting index:');
console.log('Deleting index:');

var response = await client.indices.delete({
index: index_name,
});

console.log(response.body);
```

## Create a Point in Time

```javascript
console.log('Creating a PIT:');

var response = await client.createPit({
index: 'books*',
keep_alive: '100m',
expand_wildcards: 'all',
});

console.log(response.body);
```

## Get all PITs

```javascript
console.log('Getting all PITs:');

var response = await client.getAllPits();

console.log(response.body);
```

## Delete a Point in Time

```javascript
console.log('Deleting a PIT:');

var response = await client.deletePit({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment here for how a user can get this pit_id ?

Copy link
Contributor Author

@ajygupta ajygupta Jan 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I understand this completely. A particular point in time (PIT) is denoted by a pit_id. It's like a particular index is denoted by an index_name

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we get a pit_id?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We get it when we create a point in time.

body: {
pit_id: [
'o463QQEPbXktaW5kZXgtMDAwMDAxFkhGN09fMVlPUkVPLXh6MUExZ1hpaEEAFjBGbmVEZHdGU1EtaFhhUFc4ZkR5cWcAAAAAAAAAAAEWaXBPNVJtZEhTZDZXTWFFR05waXdWZwEWSEY3T18xWU9SRU8teHoxQTFnWGloQQAA',
],
},
});

console.log(response.body);
```

## Delete all PITs

```javascript
console.log('Deleting all PITs:');

var response = await client.indices.delete({
index: index_name,
});
var response = await client.deleteAllPits();

console.log(response.body);
console.log(response.body);
```
86 changes: 86 additions & 0 deletions api/api/create_pit.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the license header for new added file should be

/*
 * Copyright OpenSearch Contributors
 * SPDX-License-Identifier: Apache-2.0
 */

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ananzh I updated as per the suggestion but the license check fails. Do we need to fix that?

Copy link
Member

@ananzh ananzh Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajygupta

I think currently opensearch-js is using this header here. Pls double check.

/*
 * Copyright OpenSearch Contributors
 * SPDX-License-Identifier: Apache-2.0
 *
 * The OpenSearch Contributors require contributions made to
 * this file be licensed under the Apache-2.0 license or a
 * compatible open source license.
 *
 */

Well, we have updated this header to the one I mentioned in the comment.
Some discussions as reference
opensearch-project/opensearch-build#2666 (comment)
opensearch-project/alerting#136 (comment)

However, it is not updated in opensearch-js and the header check is still checking the old one. To unblock you, I would suggest to use the old one and I will submit an issue or PR to update the header. Sorry for the inconvenience.

* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
*/

'use strict';

/* eslint camelcase: 0 */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the meaning of this two lines of eslint rules? could you provide some explains for adding these two lines? for example, is this eslint camelcase: 0 same as esline-disable camelcase which disable camelcase check for a file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just to keep it consistent with the other files, so that once we fix the auto-generation logic, we won't see any changes in these files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well I think I don't understand these two lines. Do you understand why we use these two lines?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK eslint camelcase: 0 and eslint no-unused-vars: 0 are for disabling camelcase check and unused variables check respectively.

/* eslint no-unused-vars: 0 */

const { handleError, snakeCaseKeys, normalizeArguments, kConfigurationError } = require('../utils');
const acceptedQuerystring = [
'allow_partial_pit_creation',
'keep_alive',
'preference',
'routing',
'pretty',
'human',
'error_trace',
'source',
'filter_path',
];
const snakeCase = {
allowPartialPitCreation: 'allow_partial_pit_creation',
keepAlive: 'keep_alive',
errorTrace: 'error_trace',
filterPath: 'filter_path',
};

/**
* Creates a point in time.
* <br/> See Also: {@link https://opensearch.org/docs/latest/opensearch/point-in-time-api#create-a-pit|Opensearch - Create a PIT}
* @memberOf API-PIT
*
* @param {Object} params
* @param {string} params.index - The name(s) of the target index(es) for the PIT. May contain a comma-separated list or a wildcard index pattern.
ajygupta marked this conversation as resolved.
Show resolved Hide resolved
ananzh marked this conversation as resolved.
Show resolved Hide resolved
* @param {string} params.keep_alive - The amount of time to keep the PIT
* @param {string} [params.preference=random] - The node or the shard used to perform the search.
* @param {string} [params.routing] - Specifies to route search requests to a specific shard.
* @param {string} [params.expand_wildcards=open] - The type of index that can match the wildcard pattern. Supports comma-separated values.
* @param {string} [params.allow_partial_pit_creation=false] - Specifies whether to create a PIT with partial failures.
*
* @param {Object} [options] - Options for {@link Transport#request}
* @param {function} [callback] - Callback that handles errors and response
*
* @returns {{abort: function(), then: function(), catch: function()}|Promise<never>|*} {@link https://opensearch.org/docs/latest/opensearch/point-in-time-api#sample-response|Create PIT Response}
*/

function createPitApi(params, options, callback) {
[params, options, callback] = normalizeArguments(params, options, callback);

// check required parameters
if (params['index'] == null) {
const err = new this[kConfigurationError]('Missing required parameter: index');
return handleError(err, callback);
}

if (params['keep_alive'] == null) {
const err = new this[kConfigurationError]('Missing required parameter: keep_alive');
return handleError(err, callback);
}

let { method, body, index, ...querystring } = params;
querystring = snakeCaseKeys(acceptedQuerystring, snakeCase, querystring);

let path = '';
if (method == null) method = 'POST';
path = '/' + encodeURIComponent(index) + '/' + '_search' + '/' + 'point_in_time';
ananzh marked this conversation as resolved.
Show resolved Hide resolved

// build request object
const request = {
method,
path,
body: body || '',
querystring,
};

return this.transport.request(request, options, callback);
}

module.exports = createPitApi;
53 changes: 53 additions & 0 deletions api/api/delete_all_pits.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/*
ajygupta marked this conversation as resolved.
Show resolved Hide resolved
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*
*/

'use strict';

/* eslint camelcase: 0 */
/* eslint no-unused-vars: 0 */

const { handleError, snakeCaseKeys, normalizeArguments, kConfigurationError } = require('../utils');
const acceptedQuerystring = ['pretty', 'human', 'error_trace', 'source', 'filter_path'];
const snakeCase = { errorTrace: 'error_trace', filterPath: 'filter_path' };

/**
* Deletes all PITs in the OpenSearch cluster. The Delete All PITs API deletes only local PITs or mixed PITs (PITs created in both local and remote clusters). It does not delete fully remote PITs.
* <br/> See Also: {@link https://opensearch.org/docs/latest/opensearch/point-in-time-api#delete-pits|Opensearch - Delete PITs}
ananzh marked this conversation as resolved.
Show resolved Hide resolved
* @memberOf API-PIT
*
* @param {Object} params
*
* @param {Object} [options] - Options for {@link Transport#request}
* @param {function} [callback] - Callback that handles errors and response
*
* @returns {{abort: function(), then: function(), catch: function()}|Promise<never>|*} {@link https://opensearch.org/docs/latest/opensearch/point-in-time-api#sample-response-2|Delete all PITs Response}
*/
function deleteAllPitsApi(params, options, callback) {
[params, options, callback] = normalizeArguments(params, options, callback);

let { method, body, ...querystring } = params;
querystring = snakeCaseKeys(acceptedQuerystring, snakeCase, querystring);

let path = '';
if (method == null) method = 'DELETE';
path = '/' + '_search' + '/' + 'point_in_time' + '/' + '_all';

// build request object
const request = {
method,
path,
body: body || '',
querystring,
};

return this.transport.request(request, options, callback);
}

module.exports = deleteAllPitsApi;
Loading