|
| 1 | +[[development-plugin-saved-objects]] |
| 2 | +== Using Saved Objects |
| 3 | + |
| 4 | +Saved Objects allow {kib} plugins to use {es} like a primary |
| 5 | +database. Think of it as an Object Document Mapper for {es}. Once a |
| 6 | +plugin has registered one or more Saved Object types, the Saved Objects client |
| 7 | +can be used to query or perform create, read, update and delete operations on |
| 8 | +each type. |
| 9 | + |
| 10 | +By using Saved Objects your plugin can take advantage of the following |
| 11 | +features: |
| 12 | + |
| 13 | +* Migrations can evolve your document's schema by transforming documents and |
| 14 | +ensuring that the field mappings on the index are always up to date. |
| 15 | +* a <<saved-objects-api,HTTP API>> is automatically exposed for each type (unless |
| 16 | +`hidden=true` is specified). |
| 17 | +* a Saved Objects client that can be used from both the server and the browser. |
| 18 | +* Users can import or export Saved Objects using the Saved Objects management |
| 19 | +UI or the Saved Objects import/export API. |
| 20 | +* By declaring `references`, an object's entire reference graph will be |
| 21 | +exported. This makes it easy for users to export e.g. a `dashboard` object and |
| 22 | +have all the `visualization` objects required to display the dashboard |
| 23 | +included in the export. |
| 24 | +* When the X-Pack security and spaces plugins are enabled these transparently |
| 25 | +provide RBAC access control and the ability to organize Saved Objects into |
| 26 | +spaces. |
| 27 | + |
| 28 | +This document contains developer guidelines and best-practices for plugins |
| 29 | +wanting to use Saved Objects. |
| 30 | + |
| 31 | +=== Registering a Saved Object type |
| 32 | +Saved object type definitions should be defined in their own `my_plugin/server/saved_objects` directory. |
| 33 | + |
| 34 | +The folder should contain a file per type, named after the snake_case name of the type, and an `index.ts` file exporting all the types. |
| 35 | + |
| 36 | +.src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts |
| 37 | +[source,typescript] |
| 38 | +---- |
| 39 | +import { SavedObjectsType } from 'src/core/server'; |
| 40 | +
|
| 41 | +export const dashboardVisualization: SavedObjectsType = { |
| 42 | + name: 'dashboard_visualization', // <1> |
| 43 | + hidden: false, |
| 44 | + namespaceType: 'single', |
| 45 | + mappings: { |
| 46 | + dynamic: false, |
| 47 | + properties: { |
| 48 | + description: { |
| 49 | + type: 'text', |
| 50 | + }, |
| 51 | + hits: { |
| 52 | + type: 'integer', |
| 53 | + }, |
| 54 | + }, |
| 55 | + }, |
| 56 | + migrations: { |
| 57 | + '1.0.0': migratedashboardVisualizationToV1, |
| 58 | + '2.0.0': migratedashboardVisualizationToV2, |
| 59 | + }, |
| 60 | +}; |
| 61 | +---- |
| 62 | +<1> Since the name of a Saved Object type forms part of the url path for the |
| 63 | +public Saved Objects HTTP API, these should follow our API URL path convention |
| 64 | +and always be written as snake case. |
| 65 | + |
| 66 | +.src/plugins/my_plugin/server/saved_objects/index.ts |
| 67 | +[source,typescript] |
| 68 | +---- |
| 69 | +export { dashboardVisualization } from './dashboard_visualization'; |
| 70 | +export { dashboard } from './dashboard'; |
| 71 | +---- |
| 72 | + |
| 73 | +.src/plugins/my_plugin/server/plugin.ts |
| 74 | +[source,typescript] |
| 75 | +---- |
| 76 | +import { dashboard, dashboardVisualization } from './saved_objects'; |
| 77 | +
|
| 78 | +export class MyPlugin implements Plugin { |
| 79 | + setup({ savedObjects }) { |
| 80 | + savedObjects.registerType(dashboard); |
| 81 | + savedObjects.registerType(dashboardVisualization); |
| 82 | + } |
| 83 | +} |
| 84 | +---- |
| 85 | + |
| 86 | +=== Mappings |
| 87 | +Each Saved Object type can define it's own {es} field mappings. |
| 88 | +Because multiple Saved Object types can share the same index, mappings defined |
| 89 | +by a type will be nested under a top-level field that matches the type name. |
| 90 | + |
| 91 | +For example, the mappings defined by the `dashboard_visualization` Saved |
| 92 | +Object type: |
| 93 | + |
| 94 | +.src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts |
| 95 | +[source,typescript] |
| 96 | +---- |
| 97 | +import { SavedObjectsType } from 'src/core/server'; |
| 98 | +
|
| 99 | +export const dashboardVisualization: SavedObjectsType = { |
| 100 | + name: 'dashboard_visualization', |
| 101 | + ... |
| 102 | + mappings: { |
| 103 | + properties: { |
| 104 | + dynamic: false, |
| 105 | + description: { |
| 106 | + type: 'text', |
| 107 | + }, |
| 108 | + hits: { |
| 109 | + type: 'integer', |
| 110 | + }, |
| 111 | + }, |
| 112 | + }, |
| 113 | + migrations: { ... }, |
| 114 | +}; |
| 115 | +---- |
| 116 | + |
| 117 | +Will result in the following mappings being applied to the `.kibana` index: |
| 118 | +[source,json] |
| 119 | +---- |
| 120 | +{ |
| 121 | + "mappings": { |
| 122 | + "dynamic": "strict", |
| 123 | + "properties": { |
| 124 | + ... |
| 125 | + "dashboard_vizualization": { |
| 126 | + "dynamic": false, |
| 127 | + "properties": { |
| 128 | + "description": { |
| 129 | + "type": "text", |
| 130 | + }, |
| 131 | + "hits": { |
| 132 | + "type": "integer", |
| 133 | + }, |
| 134 | + }, |
| 135 | + } |
| 136 | + } |
| 137 | + } |
| 138 | +} |
| 139 | +---- |
| 140 | + |
| 141 | +Do not use field mappings like you would use data types for the columns of a |
| 142 | +SQL database. Instead, field mappings are analogous to a SQL index. Only |
| 143 | +specify field mappings for the fields you wish to search on or query. By |
| 144 | +specifying `dynamic: false` in any level of your mappings, {es} will |
| 145 | +accept and store any other fields even if they are not specified in your mappings. |
| 146 | + |
| 147 | +Since {es} has a default limit of 1000 fields per index, plugins |
| 148 | +should carefully consider the fields they add to the mappings. Similarly, |
| 149 | +Saved Object types should never use `dynamic: true` as this can cause an |
| 150 | +arbitrary amount of fields to be added to the `.kibana` index. |
| 151 | + |
| 152 | +=== References |
| 153 | +When a Saved Object declares `references` to other Saved Objects, the |
| 154 | +Saved Objects Export API will automatically export the target object with all |
| 155 | +of it's references. This makes it easy for users to export the entire |
| 156 | +reference graph of an object. |
| 157 | + |
| 158 | +If a Saved Object can't be used on it's own, that is, it needs other objects |
| 159 | +to exist for a feature to function correctly, that Saved Object should declare |
| 160 | +references to all the objects it requires. For example, a `dashboard` |
| 161 | +object might have panels for several `visualization` objects. When these |
| 162 | +`visualization` objects don't exist, the dashboard cannot be rendered |
| 163 | +correctly. The `dashboard` object should declare references to all it's |
| 164 | +visualizations. |
| 165 | + |
| 166 | +However, `visualization` objects can continue to be rendered or embedded into |
| 167 | +other dashboards even if the `dashboard` it was originally embedded into |
| 168 | +doesn't exist. As a result, `visualization` objects should not declare |
| 169 | +references to `dashboard` objects. |
| 170 | + |
| 171 | +For each referenced object, an `id`, `type` and `name` are added to the |
| 172 | +`references` array: |
| 173 | + |
| 174 | +[source, typescript] |
| 175 | +---- |
| 176 | +router.get( |
| 177 | + { path: '/some-path', validate: false }, |
| 178 | + async (context, req, res) => { |
| 179 | + const object = await context.core.savedObjects.client.create( |
| 180 | + 'dashboard', |
| 181 | + { |
| 182 | + title: 'my dashboard', |
| 183 | + panels: [ |
| 184 | + { visualization: 'vis1' }, // <1> |
| 185 | + ], |
| 186 | + indexPattern: 'indexPattern1' |
| 187 | + }, |
| 188 | + { references: [ |
| 189 | + { id: '...', type: 'visualization', name: 'vis1' }, |
| 190 | + { id: '...', type: 'index_pattern', name: 'indexPattern1' }, |
| 191 | + ] |
| 192 | + } |
| 193 | + ) |
| 194 | + ... |
| 195 | + } |
| 196 | +); |
| 197 | +---- |
| 198 | +<1> Note how `dashboard.panels[0].visualization` stores the `name` property of |
| 199 | +the reference (not the `id` directly) to be able to uniquely identify this |
| 200 | +reference. This guarantees that the id the reference points to always remains |
| 201 | +up to date. If a visualization `id` was directly stored in |
| 202 | +`dashboard.panels[0].visualization` there is a risk that this `id` gets |
| 203 | +updated without updating the reference in the references array. |
| 204 | + |
| 205 | +==== Writing Migrations |
| 206 | + |
| 207 | +Saved Objects support schema changes between Kibana versions, which we call |
| 208 | +migrations. Migrations are applied when a Kibana installation is upgraded from |
| 209 | +one version to the next, when exports are imported via the Saved Objects |
| 210 | +Management UI, or when a new object is created via the HTTP API. |
| 211 | + |
| 212 | +Each Saved Object type may define migrations for its schema. Migrations are |
| 213 | +specified by the Kibana version number, receive an input document, and must |
| 214 | +return the fully migrated document to be persisted to Elasticsearch. |
| 215 | + |
| 216 | +Let's say we want to define two migrations: |
| 217 | +- In version 1.1.0, we want to drop the `subtitle` field and append it to the |
| 218 | + title |
| 219 | +- In version 1.4.0, we want to add a new `id` field to every panel with a newly |
| 220 | + generated UUID. |
| 221 | + |
| 222 | +First, the current `mappings` should always reflect the latest or "target" |
| 223 | +schema. Next, we should define a migration function for each step in the schema |
| 224 | +evolution: |
| 225 | + |
| 226 | +src/plugins/my_plugin/server/saved_objects/dashboard_visualization.ts |
| 227 | +[source,typescript] |
| 228 | +---- |
| 229 | +import { SavedObjectsType, SavedObjectMigrationFn } from 'src/core/server'; |
| 230 | +import uuid from 'uuid'; |
| 231 | +
|
| 232 | +interface DashboardVisualizationPre110 { |
| 233 | + title: string; |
| 234 | + subtitle: string; |
| 235 | + panels: Array<{}>; |
| 236 | +} |
| 237 | +interface DashboardVisualization110 { |
| 238 | + title: string; |
| 239 | + panels: Array<{}>; |
| 240 | +} |
| 241 | +
|
| 242 | +interface DashboardVisualization140 { |
| 243 | + title: string; |
| 244 | + panels: Array<{ id: string }>; |
| 245 | +} |
| 246 | +
|
| 247 | +const migrateDashboardVisualization110: SavedObjectMigrationFn< |
| 248 | + DashboardVisualizationPre110, // <1> |
| 249 | + DashboardVisualization110 |
| 250 | +> = (doc) => { |
| 251 | + const { subtitle, ...attributesWithoutSubtitle } = doc.attributes; |
| 252 | + return { |
| 253 | + ...doc, // <2> |
| 254 | + attributes: { |
| 255 | + ...attributesWithoutSubtitle, |
| 256 | + title: `${doc.attributes.title} - ${doc.attributes.subtitle}`, |
| 257 | + }, |
| 258 | + }; |
| 259 | +}; |
| 260 | +
|
| 261 | +const migrateDashboardVisualization140: SavedObjectMigrationFn< |
| 262 | + DashboardVisualization110, |
| 263 | + DashboardVisualization140 |
| 264 | +> = (doc) => { |
| 265 | + const outPanels = doc.attributes.panels?.map((panel) => { |
| 266 | + return { ...panel, id: uuid.v4() }; |
| 267 | + }); |
| 268 | + return { |
| 269 | + ...doc, |
| 270 | + attributes: { |
| 271 | + ...doc.attributes, |
| 272 | + panels: outPanels, |
| 273 | + }, |
| 274 | + }; |
| 275 | +}; |
| 276 | +
|
| 277 | +export const dashboardVisualization: SavedObjectsType = { |
| 278 | + name: 'dashboard_visualization', // <1> |
| 279 | + /** ... */ |
| 280 | + migrations: { |
| 281 | + // Takes a pre 1.1.0 doc, and converts it to 1.1.0 |
| 282 | + '1.1.0': migrateDashboardVisualization110, |
| 283 | +
|
| 284 | + // Takes a 1.1.0 doc, and converts it to 1.4.0 |
| 285 | + '1.4.0': migrateDashboardVisualization140, // <3> |
| 286 | + }, |
| 287 | +}; |
| 288 | +---- |
| 289 | +<1> It is useful to define an interface for each version of the schema. This |
| 290 | +allows TypeScript to ensure that you are properly handling the input and output |
| 291 | +types correctly as the schema evolves. |
| 292 | +<2> Returning a shallow copy is necessary to avoid type errors when using |
| 293 | +different types for the input and output shape. |
| 294 | +<3> Migrations do not have to be defined for every version. The version number |
| 295 | +of a migration must always be the earliest Kibana version in which this |
| 296 | +migration was released. So if you are creating a migration which will be |
| 297 | +part of the v7.10.0 release, but will also be backported and released as |
| 298 | +v7.9.3, the migration version should be: 7.9.3. |
| 299 | + |
| 300 | +Migrations should be written defensively, an exception in a migration function |
| 301 | +will prevent a Kibana upgrade from succeeding and will cause downtime for our |
| 302 | +users. Having said that, if a document is encountered that is not in the |
| 303 | +expected shape, migrations are encouraged to throw an exception to abort the |
| 304 | +upgrade. In most scenarios, it is better to fail an upgrade than to silently |
| 305 | +ignore a corrupt document which can cause unexpected behaviour at some future |
| 306 | +point in time. |
| 307 | + |
| 308 | +It is critical that you have extensive tests to ensure that migrations behave |
| 309 | +as expected with all possible input documents. Given how simple it is to test |
| 310 | +all the branch conditions in a migration function and the high impact of a bug |
| 311 | +in this code, there's really no reason not to aim for 100% test code coverage. |
0 commit comments