Migrate visualisations to Lens

[ruflin]

Lens is the main editor that should be used for visualisations. Existing dashboards we own must be migrated to it. In #265 a previous discussion happened on Gaps to moving to Lens. The obvious ones are resolved but there might be more we uncover during the migration. This issue has the purpose to uncover these issues and track the migration to Lens. Any blockers during the migration must be mentioned in this issue. The current state of the migration from legacy visualisations to Lens for integrations can be found here (private link).

On overview from the state of 2022-12-06 is shared on the bottom. For tracking, I will list the top 5:

• MIgrate all AWS visualisations to lens #4994
• System @elastic/elastic-agent-data-plane
• Cef @elastic/security-external-integrations
• Migrate all Azure visualizations to lens #4995
• MIgrate all GCP dashboards to lens #5015
• Network Traffic @elastic/security-external-integrations

For the migration, there is an "Open in Lens" button which should hopefully help with the migration. The overall goal is to migrate all visualisations but lets start with the top 5 as the initial priority to also uncover potential migration blockers.

[ruflin]

@jsoriano Similar discussion applies here like for by reference. Can we add some hints during the build process or similar to warn devs?

[jsoriano]

Similar discussion applies here like for by reference. Can we add some hints during the build process or similar to warn devs?

We need some better way to encourage developers to use more current standards. Printing warnings as we do now in some cases is easily ignored or overlooked. Producing errors in CI is usually too disrupting and difficult to introduce. We'd need to find some intermediary approach. Maybe elastic/package-spec#342 can help, but is probably not enough.

We may need some mechanism to encourage warnings reduction in CI, maybe we can do something like failing if a PR affecting a package with warnings doesn't reduce the number of them.

[ruflin]

@jsoriano Agree. This might become a longer discussion, lets take it to a separate issue.

[drewdaemon]

@gvnmagni is part-way through a Lens-based refactor of the system metrics dash!

[kaiyan-sheng]

Which version of Kibana starts supporting "Open in Lens" button?

[drewdaemon]

@kaiyan-sheng it was first introduced in 8.2. We have continued adding support for further configurations since then, especially in 8.5 and 8.6 when we finished all the cases we outlined for phase 1. This is the overall meta-issue that links to everything.

So, a gradually increasing number of legacy visualizations are convertible as you advance from 8.2 forward to 8.6 where the vast majority can be (still subject to human review).

In 8.7, we'll be promoting "Open in Lens" on dashboards (you can see some screenshots on elastic/kibana#146363) which will make it more obvious which visualizations can be automatically converted and easier to convert them.

[ruflin]

One challenge we will have during converting to Lens is that many / most of the packages are backward compatible with 7.17 which means there are limits on the features that 7.17 has. At some point, the compatibility version needs to be increased of the package so we can make use of all the features in Lens.

[milanparmar-crest]

Team FYI, We're currently scoping this effort and we are planning to work on this issue.

[roshan-elastic]

We have a load of visualisations in the hosts view which would benefit from moving to Lens (I had an SRE interview today where they were expecting to be able to go into lens and add):

+1 on the effort (you can assume the Infra UI Team will prioritise this effort where it adds business value - I'm seeing this already)

[milanparmar-crest]

Created an issue to track the Observability integrations on which we are working to migrate in Lens. #5221