Skip to content

Commit b24dcdb

Browse files
elasticmachineelasticmachine
authored
Merge branch 'master' into ml-exernal-url-replace-hash
2 parents 6da3cb3 + 8547b32 commit b24dcdb

File tree

494 files changed

+12930
-2815
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

494 files changed

+12930
-2815
lines changed

‎docs/developer/architecture/development-plugin-saved-objects.asciidoc‎

Lines changed: 311 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,311 @@
1+
[[development-plugin-saved-objects]]
2+
== Using Saved Objects
3+
4+
Saved Objects allow {kib} plugins to use {es} like a primary
5+
database. Think of it as an Object Document Mapper for {es}. Once a
6+
plugin has registered one or more Saved Object types, the Saved Objects client
7+
can be used to query or perform create, read, update and delete operations on
8+
each type.
9+
10+
By using Saved Objects your plugin can take advantage of the following
11+
features:
12+
13+
* Migrations can evolve your document's schema by transforming documents and
14+
ensuring that the field mappings on the index are always up to date.
15+
* a <<saved-objects-api,HTTP API>> is automatically exposed for each type (unless
16+
`hidden=true` is specified).
17+
* a Saved Objects client that can be used from both the server and the browser.
18+
* Users can import or export Saved Objects using the Saved Objects management
19+
UI or the Saved Objects import/export API.
20+
* By declaring `references`, an object's entire reference graph will be
21+
exported. This makes it easy for users to export e.g. a `dashboard` object and
22+
have all the `visualization` objects required to display the dashboard
23+
included in the export.
24+
* When the X-Pack security and spaces plugins are enabled these transparently
25+
provide RBAC access control and the ability to organize Saved Objects into
26+
spaces.
27+
28+
This document contains developer guidelines and best-practices for plugins
29+
wanting to use Saved Objects.
30+
31+
=== Registering a Saved Object type
32+
Saved object type definitions should be defined in their own `my_plugin/server/saved_objects` directory.
33+
34+
The folder should contain a file per type, named after the snake_case name of the type, and an `index.ts` file exporting all the types.
35+
36+
.src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts
37+
[source,typescript]
38+
----
39+
import { SavedObjectsType } from 'src/core/server';
40+
41+
export const dashboardVisualization: SavedObjectsType = {
42+
name: 'dashboard_visualization', // <1>
43+
hidden: false,
44+
namespaceType: 'single',
45+
mappings: {
46+
dynamic: false,
47+
properties: {
48+
description: {
49+
type: 'text',
50+
},
51+
hits: {
52+
type: 'integer',
53+
},
54+
},
55+
},
56+
migrations: {
57+
'1.0.0': migratedashboardVisualizationToV1,
58+
'2.0.0': migratedashboardVisualizationToV2,
59+
},
60+
};
61+
----
62+
<1> Since the name of a Saved Object type forms part of the url path for the
63+
public Saved Objects HTTP API, these should follow our API URL path convention
64+
and always be written as snake case.
65+
66+
.src/plugins/my_plugin/server/saved_objects/index.ts
67+
[source,typescript]
68+
----
69+
export { dashboardVisualization } from './dashboard_visualization';
70+
export { dashboard } from './dashboard';
71+
----
72+
73+
.src/plugins/my_plugin/server/plugin.ts
74+
[source,typescript]
75+
----
76+
import { dashboard, dashboardVisualization } from './saved_objects';
77+
78+
export class MyPlugin implements Plugin {
79+
setup({ savedObjects }) {
80+
savedObjects.registerType(dashboard);
81+
savedObjects.registerType(dashboardVisualization);
82+
}
83+
}
84+
----
85+
86+
=== Mappings
87+
Each Saved Object type can define it's own {es} field mappings.
88+
Because multiple Saved Object types can share the same index, mappings defined
89+
by a type will be nested under a top-level field that matches the type name.
90+
91+
For example, the mappings defined by the `dashboard_visualization` Saved
92+
Object type:
93+
94+
.src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts
95+
[source,typescript]
96+
----
97+
import { SavedObjectsType } from 'src/core/server';
98+
99+
export const dashboardVisualization: SavedObjectsType = {
100+
name: 'dashboard_visualization',
101+
...
102+
mappings: {
103+
properties: {
104+
dynamic: false,
105+
description: {
106+
type: 'text',
107+
},
108+
hits: {
109+
type: 'integer',
110+
},
111+
},
112+
},
113+
migrations: { ... },
114+
};
115+
----
116+
117+
Will result in the following mappings being applied to the `.kibana` index:
118+
[source,json]
119+
----
120+
{
121+
"mappings": {
122+
"dynamic": "strict",
123+
"properties": {
124+
...
125+
"dashboard_vizualization": {
126+
"dynamic": false,
127+
"properties": {
128+
"description": {
129+
"type": "text",
130+
},
131+
"hits": {
132+
"type": "integer",
133+
},
134+
},
135+
}
136+
}
137+
}
138+
}
139+
----
140+
141+
Do not use field mappings like you would use data types for the columns of a
142+
SQL database. Instead, field mappings are analogous to a SQL index. Only
143+
specify field mappings for the fields you wish to search on or query. By
144+
specifying `dynamic: false` in any level of your mappings, {es} will
145+
accept and store any other fields even if they are not specified in your mappings.
146+
147+
Since {es} has a default limit of 1000 fields per index, plugins
148+
should carefully consider the fields they add to the mappings. Similarly,
149+
Saved Object types should never use `dynamic: true` as this can cause an
150+
arbitrary amount of fields to be added to the `.kibana` index.
151+
152+
=== References
153+
When a Saved Object declares `references` to other Saved Objects, the
154+
Saved Objects Export API will automatically export the target object with all
155+
of it's references. This makes it easy for users to export the entire
156+
reference graph of an object.
157+
158+
If a Saved Object can't be used on it's own, that is, it needs other objects
159+
to exist for a feature to function correctly, that Saved Object should declare
160+
references to all the objects it requires. For example, a `dashboard`
161+
object might have panels for several `visualization` objects. When these
162+
`visualization` objects don't exist, the dashboard cannot be rendered
163+
correctly. The `dashboard` object should declare references to all it's
164+
visualizations.
165+
166+
However, `visualization` objects can continue to be rendered or embedded into
167+
other dashboards even if the `dashboard` it was originally embedded into
168+
doesn't exist. As a result, `visualization` objects should not declare
169+
references to `dashboard` objects.
170+
171+
For each referenced object, an `id`, `type` and `name` are added to the
172+
`references` array:
173+
174+
[source, typescript]
175+
----
176+
router.get(
177+
{ path: '/some-path', validate: false },
178+
async (context, req, res) => {
179+
const object = await context.core.savedObjects.client.create(
180+
'dashboard',
181+
{
182+
title: 'my dashboard',
183+
panels: [
184+
{ visualization: 'vis1' }, // <1>
185+
],
186+
indexPattern: 'indexPattern1'
187+
},
188+
{ references: [
189+
{ id: '...', type: 'visualization', name: 'vis1' },
190+
{ id: '...', type: 'index_pattern', name: 'indexPattern1' },
191+
]
192+
}
193+
)
194+
...
195+
}
196+
);
197+
----
198+
<1> Note how `dashboard.panels[0].visualization` stores the `name` property of
199+
the reference (not the `id` directly) to be able to uniquely identify this
200+
reference. This guarantees that the id the reference points to always remains
201+
up to date. If a visualization `id` was directly stored in
202+
`dashboard.panels[0].visualization` there is a risk that this `id` gets
203+
updated without updating the reference in the references array.
204+
205+
==== Writing Migrations
206+
207+
Saved Objects support schema changes between Kibana versions, which we call
208+
migrations. Migrations are applied when a Kibana installation is upgraded from
209+
one version to the next, when exports are imported via the Saved Objects
210+
Management UI, or when a new object is created via the HTTP API.
211+
212+
Each Saved Object type may define migrations for its schema. Migrations are
213+
specified by the Kibana version number, receive an input document, and must
214+
return the fully migrated document to be persisted to Elasticsearch.
215+
216+
Let's say we want to define two migrations:
217+
- In version 1.1.0, we want to drop the `subtitle` field and append it to the
218+
title
219+
- In version 1.4.0, we want to add a new `id` field to every panel with a newly
220+
generated UUID.
221+
222+
First, the current `mappings` should always reflect the latest or "target"
223+
schema. Next, we should define a migration function for each step in the schema
224+
evolution:
225+
226+
src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts
227+
[source,typescript]
228+
----
229+
import { SavedObjectsType, SavedObjectMigrationFn } from 'src/core/server';
230+
import uuid from 'uuid';
231+
232+
interface DashboardVisualizationPre110 {
233+
title: string;
234+
subtitle: string;
235+
panels: Array<{}>;
236+
}
237+
interface DashboardVisualization110 {
238+
title: string;
239+
panels: Array<{}>;
240+
}
241+
242+
interface DashboardVisualization140 {
243+
title: string;
244+
panels: Array<{ id: string }>;
245+
}
246+
247+
const migrateDashboardVisualization110: SavedObjectMigrationFn<
248+
DashboardVisualizationPre110, // <1>
249+
DashboardVisualization110
250+
> = (doc) => {
251+
const { subtitle, ...attributesWithoutSubtitle } = doc.attributes;
252+
return {
253+
...doc, // <2>
254+
attributes: {
255+
...attributesWithoutSubtitle,
256+
title: `${doc.attributes.title} - ${doc.attributes.subtitle}`,
257+
},
258+
};
259+
};
260+
261+
const migrateDashboardVisualization140: SavedObjectMigrationFn<
262+
DashboardVisualization110,
263+
DashboardVisualization140
264+
> = (doc) => {
265+
const outPanels = doc.attributes.panels?.map((panel) => {
266+
return { ...panel, id: uuid.v4() };
267+
});
268+
return {
269+
...doc,
270+
attributes: {
271+
...doc.attributes,
272+
panels: outPanels,
273+
},
274+
};
275+
};
276+
277+
export const dashboardVisualization: SavedObjectsType = {
278+
name: 'dashboard_visualization', // <1>
279+
/** ... */
280+
migrations: {
281+
// Takes a pre 1.1.0 doc, and converts it to 1.1.0
282+
'1.1.0': migrateDashboardVisualization110,
283+
284+
// Takes a 1.1.0 doc, and converts it to 1.4.0
285+
'1.4.0': migrateDashboardVisualization140, // <3>
286+
},
287+
};
288+
----
289+
<1> It is useful to define an interface for each version of the schema. This
290+
allows TypeScript to ensure that you are properly handling the input and output
291+
types correctly as the schema evolves.
292+
<2> Returning a shallow copy is necessary to avoid type errors when using
293+
different types for the input and output shape.
294+
<3> Migrations do not have to be defined for every version. The version number
295+
of a migration must always be the earliest Kibana version in which this
296+
migration was released. So if you are creating a migration which will be
297+
part of the v7.10.0 release, but will also be backported and released as
298+
v7.9.3, the migration version should be: 7.9.3.
299+
300+
Migrations should be written defensively, an exception in a migration function
301+
will prevent a Kibana upgrade from succeeding and will cause downtime for our
302+
users. Having said that, if a document is encountered that is not in the
303+
expected shape, migrations are encouraged to throw an exception to abort the
304+
upgrade. In most scenarios, it is better to fail an upgrade than to silently
305+
ignore a corrupt document which can cause unexpected behaviour at some future
306+
point in time.
307+
308+
It is critical that you have extensive tests to ensure that migrations behave
309+
as expected with all possible input documents. Given how simple it is to test
310+
all the branch conditions in a migration function and the high impact of a bug
311+
in this code, there's really no reason not to aim for 100% test code coverage.

‎docs/developer/architecture/index.asciidoc‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,16 @@ A few services also automatically generate api documentation which can be browse
1515
A few notable services are called out below.
1616

1717
* <<development-security>>
18+
* <<development-plugin-saved-objects>>
1819
* <<add-data-tutorials>>
1920
* <<development-visualize-index>>
2021

22+
include::security/index.asciidoc[leveloffset=+1]
23+
24+
include::development-plugin-saved-objects.asciidoc[leveloffset=+1]
25+
2126
include::add-data-tutorials.asciidoc[leveloffset=+1]
2227

2328
include::development-visualize-index.asciidoc[leveloffset=+1]
2429

25-
include::security/index.asciidoc[leveloffset=+1]
2630

‎docs/getting-started/tutorial-define-index.asciidoc‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,5 +47,11 @@ contains the time series data.
4747
[role="screenshot"]
4848
image::images/tutorial_index_patterns.png[All tutorial index patterns]
4949

50+
NOTE: When you define an index pattern, the indices that match that pattern must
51+
exist in Elasticsearch and they must contain data. To check if the indices are
52+
available, open the menu, go to *Dev Tools > Console*, then enter `GET _cat/indices`. Alternately, use
53+
`curl -XGET "http://localhost:9200/_cat/indices"`.
54+
For Windows, run `Invoke-RestMethod -Uri "http://localhost:9200/_cat/indices"` in Powershell.
55+
5056

5157

‎docs/getting-started/tutorial-full-experience.asciidoc‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,14 @@ curl -O https://download.elastic.co/demos/kibana/gettingstarted/8.x/shakespeare.
2525
curl -O https://download.elastic.co/demos/kibana/gettingstarted/8.x/accounts.zip
2626
curl -O https://download.elastic.co/demos/kibana/gettingstarted/8.x/logs.jsonl.gz
2727

28-
Two of the data sets are compressed. To extract the files, use the following commands:
28+
Alternatively, for Windows users, run the following commands in Powershell:
29+
30+
[source,shell]
31+
Invoke-RestMethod https://download.elastic.co/demos/kibana/gettingstarted/8.x/shakespeare.json -OutFile shakespeare.json
32+
Invoke-RestMethod https://download.elastic.co/demos/kibana/gettingstarted/8.x/accounts.zip -OutFile accounts.zip
33+
Invoke-RestMethod https://download.elastic.co/demos/kibana/gettingstarted/8.x/logs.jsonl.gz -OutFile logs.jsonl.gz
34+
35+
Two of the data sets are compressed. To extract the files, use these commands:
2936

3037
[source,shell]
3138
unzip accounts.zip

‎docs/user/alerting/action-types.asciidoc‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,19 @@ a| <<email-action-type, Email>>
1111

1212
| Send email from your server.
1313

14+
a| <<resilient-action-type, IBM Resilient>>
15+
16+
| Create an incident in IBM Resilient.
17+
1418
a| <<index-action-type, Index>>
1519

1620
| Index data into Elasticsearch.
1721

22+
a| <<jira-action-type, Jira>>
23+
24+
| Create an incident in Jira.
25+
26+
1827
a| <<pagerduty-action-type, PagerDuty>>
1928

2029
| Send an event in PagerDuty.
@@ -53,10 +62,12 @@ before {kib} starts.
5362
If you preconfigure a connector, you can also <<preconfigured-action-type-example, preconfigure its action type>>.
5463

5564
include::action-types/email.asciidoc[]
65+
include::action-types/resilient.asciidoc[]
5666
include::action-types/index.asciidoc[]
67+
include::action-types/jira.asciidoc[]
5768
include::action-types/pagerduty.asciidoc[]
5869
include::action-types/server-log.asciidoc[]
70+
include::action-types/servicenow.asciidoc[]
5971
include::action-types/slack.asciidoc[]
6072
include::action-types/webhook.asciidoc[]
6173
include::action-types/pre-configured-connectors.asciidoc[]
62-
include::action-types/servicenow.asciidoc[]

0 commit comments

Comments
 (0)