Use the recommended Saved Objects Export API for exporting dashboards in elastic-package

[shmsr]

During a dashboard demo presented by @lucian-ioan yesterday, he mentioned some limitations where data views are not getting exported properly.

So out of curiosity, started investigating the issue and examining how the export and import of dashboards are handled, I noticed that the Elastic Package (EP) is using the deprecated export dashboard API to export dashboards. The recommended approach is to use the generic Saved Objects export API, where the dashboard is one of the objects. The recommended API details are as follows:

See (EP using deprecated API):

elastic-package/internal/kibana/dashboards.go

Line 36 in 3957a10

path := fmt.Sprintf("%s/dashboards/export%s", CoreAPI, query.String())

Recommended API
HTTP Method: POST
API: /api/saved_objects/_export
Request body: {"objects":[{"id":"","type":"dashboard"}],"includeReferencesDeep":true}

I also observed that the export functionality in the Kibana UI is using the recommended API, as evident from the Chrome Dev Tools Network Tab.

Furthermore, I found a reference to the new API being used in the Beats repository: https://github.com/elastic/beats/blob/4496e21d4aeaa7d409a7c32eb68477ea630a1ce4/libbeat/dashboards/get.go#L48-L48

To reproduce the issue, I created a data view that points to logs-*,metrics-* and used it in the dashboard. The data view JSON is as follows:

{"attributes":{"allowHidden":false,"fieldAttrs":"{}","fieldFormatMap":"{}","fields":"[]","name":"logmet","runtimeFieldMap":"{}","sourceFilters":"[]","timeFieldName":"@timestamp","title":"logs-*,metrics-*"},"coreMigrationVersion":"8.8.0","created_at":"2024-06-19T18:13:35.075Z","created_by":"u_mddsaOF_q5bmFCATbLXAcCfggsvONAwSruelyKA5E_0","id":"0eb74d2d-e8db-4a12-3300-74aedbcc8480","managed":false,"references":[],"type":"index-pattern","typeMigrationVersion":"8.0.0","updated_at":"2024-06-19T18:13:35.075Z","version":"WzQxLDFd"}

However, when using EP's export dashboard functionality, the above-mentioned data-view was not included in the exported dashboard JSON, which explains why data views have to be manually created?

It's worth noting that the response of the new API is in NDJSON format. I took a look at the Kibana codebase, and it seems like it should work as I checked the parts of the installation of packages in fleet plugin; but I'm not an expert in TypeScript, so I cannot say with certainty if they using the saved objects import API.

We would like to open a discussion on whether we should migrate to the new Saved Objects API in EP for exporting dashboards. Should we?

[jsoriano]

We could revive #1357 for this.

[shmsr]

We could revive #1357 for this.

Ah, you already have it 💯

Do you think this part should also go in?

[jsoriano]

Do you think this part should also go in?

I think we are already filtering out index patterns:

elastic-package/internal/export/transform_filter.go

Line 14 in 87e0d1a

case "index-pattern": // unsupported types

[shmsr]

Oh, right. Missed that. Thanks!