Skip to content

Commit 95f6b4f

Browse files
Aschenscottinet
Aschen
authored and
scottinet
committed
Add bulk:write and bulk:mWrite (#419)
## What does this PR do? Add `bulk:write` and `bulk:mWrite` support. I'm not sure if it's an enhancement or a new feature (it was a new feature in Kuzzle)
1 parent 877fc42 commit 95f6b4f

File tree

8 files changed

+360
-4
lines changed

8 files changed

+360
-4
lines changed

‎doc/6/controllers/bulk/mWrite/index.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
code: true
3+
type: page
4+
title: mWrite
5+
description: Creates or replaces multiple documents directly into the storage engine.
6+
---
7+
8+
# mWrite
9+
10+
<SinceBadge version="6.2.0" />
11+
12+
<SinceBadge version="Kuzzle 1.8.0" />
13+
14+
Creates or replaces multiple documents directly into the storage engine.
15+
16+
This is a low level route intended to bypass Kuzzle actions on document creation, notably:
17+
- check [document validity](/core/1/guides/essentials/data-validation),
18+
- add [kuzzle metadata](/core/1/guides/essentials/document-metadata),
19+
- trigger [realtime notifications](/core/1/guides/essentials/real-time) (unless asked otherwise)
20+
21+
<br/>
22+
23+
```js
24+
mWrite (index, collection, documents, [options])
25+
```
26+
27+
<br/>
28+
29+
| Argument | Type | Description |
30+
| ------------ | ----------------- | -------------------- |
31+
| `index` | <pre>string</pre> | Index name |
32+
| `collection` | <pre>string</pre> | Collection name |
33+
| `documents` | <pre>object[]</pre> | Array of objects representing the documents |
34+
| `options` | <pre>object</pre> | Query options |
35+
36+
### documents
37+
38+
An array of objects. Each object describes a document to create or replace, by exposing the following properties:
39+
- `_id`: document unique identifier (optional)
40+
- `body`: document content
41+
42+
### options
43+
44+
Additional query options
45+
46+
| Property | Type<br/>(default) | Description |
47+
| ---------- | ------------------------------- | ------------------ |
48+
| `queuable` | <pre>boolean</pre><br/>(`true`) | If true, queues the request during downtime, until connected to Kuzzle again |
49+
| `notify` | <pre>boolean</pre><br/>(`false`) | if set to true, Kuzzle will trigger realtime notifications |
50+
| `refresh` | <pre>string</pre><br/>(`""`) | If set to `wait_for`, waits for the change to be reflected for `search` (up to 1s) |
51+
52+
---
53+
54+
## Resolves
55+
56+
Returns a `hits` array, containing the list of created documents, in the same order than the one provided in the query.
57+
58+
Each created document is an object with the following properties:
59+
60+
| Name | Type | Description |
61+
| --------- | ----------------- | ------------------- |
62+
| `_id` | <pre>string</pre> | ID of the newly created document |
63+
| `_version` | <pre>number</pre> | Version of the document in the persistent data storage |
64+
| `_source` | <pre>object</pre> | Created document |
65+
66+
If one or more document creations fail, the promise is rejected and the `error` object contains a [partial error](/core/1/api/essentials/errors/#partialerror) error.
67+
68+
## Usage
69+
70+
<<< ./snippets/mWrite.js

‎doc/6/controllers/bulk/mWrite/snippets/mWrite.js‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
try {
2+
const documents = [
3+
{
4+
_id: 'panipokari',
5+
body: {
6+
licence: 'B',
7+
_kuzzle_info: { creator: 'bahi', createdAt: 746186305 }
8+
}
9+
},
10+
{
11+
body: {
12+
licence: 'B',
13+
_kuzzle_info: { creator: 'dahi', createdAt: 835005505 }
14+
}
15+
}
16+
];
17+
18+
const result = await kuzzle.bulk.mWrite(
19+
'nyc-open-data',
20+
'yellow-taxi',
21+
documents
22+
);
23+
24+
console.log(result);
25+
/*
26+
{ hits:
27+
[ { _id: 'panipokari',
28+
_source: {
29+
"licence": "B",
30+
"_kuzzle_info": {
31+
"creator": "bahi",
32+
"createdAt": 746186305 } },
33+
_version: 1 },
34+
{ _id: 'AWvlrMIZE7sKlamkyoHr',
35+
_source: {
36+
"licence": "B",
37+
"_kuzzle_info": {
38+
"creator": "dahi",
39+
"createdAt": 835005505 } },
40+
_version: 1 } ],
41+
total: 2 }
42+
*/
43+
44+
console.log(`Document creator is ${result.hits[1]._source._kuzzle_info.creator}`);
45+
} catch (error) {
46+
console.error(error.message);
47+
}

‎doc/6/controllers/bulk/mWrite/snippets/mWrite.test.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
name: bulk#mWrite
3+
description: Creates or replaces multiple documents directly into the storage engine.
4+
hooks:
5+
before: |
6+
curl -X DELETE kuzzle:7512/nyc-open-data
7+
curl -X POST kuzzle:7512/nyc-open-data/_create
8+
curl -X PUT kuzzle:7512/nyc-open-data/yellow-taxi
9+
template: default
10+
expected: Document creator is dahi

‎doc/6/controllers/bulk/write/index.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
code: true
3+
type: page
4+
title: write
5+
description: Creates or replaces a document directly into the storage engine.
6+
---
7+
8+
# write
9+
10+
<SinceBadge version="6.2.0" />
11+
12+
<SinceBadge version="Kuzzle 1.8.0" />
13+
14+
Creates or replaces a document directly into the storage engine.
15+
16+
This is a low level route intended to bypass Kuzzle actions on document creation, notably:
17+
- check [document validity](/core/1/guides/essentials/data-validation),
18+
- add [kuzzle metadata](/core/1/guides/essentials/document-metadata),
19+
- trigger [realtime notifications](/core/1/guides/essentials/real-time) (unless asked otherwise).
20+
21+
<br/>
22+
23+
```js
24+
write (index, collection, document, [id], [options])
25+
```
26+
27+
<br/>
28+
29+
| Argument | Type | Description |
30+
| ------------ | ----------------- | -------------------- |
31+
| `index` | <pre>string</pre> | Index name |
32+
| `collection` | <pre>string</pre> | Collection name |
33+
| `document` | <pre>object</pre> | Document content |
34+
| `id` | <pre>string</pre> | Optional document ID |
35+
| `options` | <pre>object</pre> | Query options |
36+
37+
### options
38+
39+
Additional query options
40+
41+
| Property | Type<br/>(default) | Description |
42+
| ---------- | ------------------------------- | ------------------ |
43+
| `queuable` | <pre>boolean</pre><br/>(`true`) | If true, queues the request during downtime, until connected to Kuzzle again |
44+
| `notify` | <pre>boolean</pre><br/>(`false`) | if set to true, Kuzzle will trigger realtime notifications |
45+
| `refresh` | <pre>string</pre><br/>(`""`) | If set to `wait_for`, waits for the change to be reflected for `search` (up to 1s) |
46+
47+
---
48+
49+
## Resolves
50+
51+
Resolves to an object containing the document creation result.
52+
53+
| Name | Type | Description |
54+
| --------- | ----------------- | ------------------------------------------------------ |
55+
| `_id` | <pre>string</pre> | ID of the newly created document |
56+
| `_version` | <pre>number</pre> | Version of the document in the persistent data storage |
57+
| `_source` | <pre>object</pre> | Created document |
58+
59+
## Usage
60+
61+
<<< ./snippets/write.js

‎doc/6/controllers/bulk/write/snippets/write.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
try {
2+
const document = {
3+
licence: 'B',
4+
_kuzzle_info: {
5+
creator: 'liia',
6+
createdAt: 653132233
7+
}
8+
};
9+
10+
const result = await kuzzle.bulk.write(
11+
'nyc-open-data',
12+
'yellow-taxi',
13+
document
14+
);
15+
16+
console.log(result);
17+
/*
18+
{ _id: 'AWvljct6E7sKlamkyoHl',
19+
_version: 1,
20+
_source:
21+
{ licence: 'B',
22+
_kuzzle_info: { creator: 'liia', createdAt: 653132233 } }
23+
*/
24+
25+
console.log(`Document creator is ${result._source._kuzzle_info.creator}`);
26+
} catch (error) {
27+
console.error(error.message);
28+
}

‎doc/6/controllers/bulk/write/snippets/write.test.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
name: bulk#write
3+
description: Creates or replaces a document directly into the storage engine.
4+
hooks:
5+
before: |
6+
curl -X DELETE kuzzle:7512/nyc-open-data
7+
curl -X POST kuzzle:7512/nyc-open-data/_create
8+
curl -X PUT kuzzle:7512/nyc-open-data/yellow-taxi
9+
template: default
10+
expected: Document creator is liia

‎src/controllers/bulk.js‎

Lines changed: 51 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,15 @@ class BulkController extends BaseController {
55
super(kuzzle, 'bulk');
66
}
77

8-
import (data, options) {
8+
/**
9+
* Creates, updates or deletes large amounts of documents as fast as possible.
10+
* {@link https://docs.kuzzle.io/core/1/api/controllers/bulk/import/|Official documentation}
11+
*
12+
* @param {Object[]} data - Array of documents detailing the bulk operations to perform, following ElasticSearch Bulk API
13+
* @param {Object} [options] - Additional options
14+
* @returns {Promise}
15+
*/
16+
import (data, options = {}) {
917
return this.query({
1018
action: 'import',
1119
body: {
@@ -15,6 +23,48 @@ class BulkController extends BaseController {
1523
.then(response => response.result);
1624
}
1725

26+
/**
27+
* Creates or replaces a document directly into the storage engine.
28+
* {@link https://docs.kuzzle.io/core/1/api/controllers/bulk/write/|Official documentation}
29+
*
30+
* @param {String} index - Index name
31+
* @param {String} collection - Collection name
32+
* @param {Object} document - Document body
33+
* @param {String} [id=null] - Document ID
34+
* @param {Object} [options] - Additional options (notify, refresh)
35+
* @returns {Promise}
36+
*/
37+
write (index, collection, document, id = null, options = {}) {
38+
return this.query({
39+
index,
40+
collection,
41+
_id: id,
42+
action: 'write',
43+
body: document
44+
}, options)
45+
.then(response => response.result);
46+
}
47+
48+
/**
49+
* Creates or replaces multiple documents directly into the storage engine.
50+
* {@link https://docs.kuzzle.io/core/1/api/controllers/bulk/m-write/|Official documentation}
51+
*
52+
* @param {String} index - Index name
53+
* @param {String} collection - Collection name
54+
* @param {Object[]} documents - Array of objects describing the documents with '_id' and '_source' properties
55+
* @param {Object} [options] - Additional options (notify, refresh)
56+
* @returns {Promise}
57+
*/
58+
mWrite (index, collection, documents, options = {}) {
59+
return this.query({
60+
index,
61+
collection,
62+
action: 'mWrite',
63+
body: { documents }
64+
}, options)
65+
.then(response => response.result);
66+
}
67+
1868
}
1969

2070
module.exports = BulkController;

0 commit comments

Comments
 (0)