Telemetry for Dashboard

[majagrubic]

With the new time to visualize, improvements and simplifications of the dashboard, we want users to create their dashboard from within a dashboard.

Metrics we would like to collect

• Number of panels created from the dashboard in the last 90 days, 30 days, and all time
• Number of visualizations saved from the visualize app in the last 90 days, 30 days, and all time
  These two metrics will allow to know:
• ratio of clusters who are creating panels from a dashboard
• % of panels created from Dashboard vs all

Currently collected

• Number of new dashboards created in the last 90 days, 30 days, and all time
• Number of visualizations per dashboard

Other metrics we may want to consider

• Number of existing dashboards edited
• Number of new visualizations created from the dashboard (total)
• Number of new visualizations created from the add panel
• Number of new visualizations created from the add button
• Number of existing visualizations added
• Breakdown of number of visualizations on dashboard
• Number of Dashboards shared

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[AlonaNadler]

I change above a few things.
In addition, it will be good to capture
Total number of new dashboard in the last 90 days and 30 days

regarding

Breakdown of number of visualizations on dashboard

Not sure how its best to be capture since its a per cluster collection and a cluster might have dozens of dashboards.
Maybe max visualizations on a dashboard, min visualizations on a dashboard and average visualizations on a dashboard

[timroes]

Not sure if this is tracked via that issue, so wanted to drop this here: Once visualizations are embedded in Dashboards directly we'll lose the existing Telemetry on them. So we basically have two options:

Dashboard specific collectors

Write telemetry collectors specifically for dashboard, that will load the dashboards and crap whatever information we need from them to report to telemetry.

Advantages:

• Easier to implement

Disadvantages:

• The telemetry data between dashboards embedded visualizations/Lens and regular ones could always be different.
• Dashboard suddenly needs to have knowledge about internal state of individual embeddables, e.g. to report which visualization types have been embedded, it needs to know where the Lens "saved object" and the visualization "saved object" store that internally, so it can write a collector to get that information from there.
• Telemetry is no longer in control of the individual plugins, but suddenly in control of the dashboard app, e.g. if ML writes an inline embeddable and want telemetry data about it, we need to extend our dashboard telemetry to gather that information.
• Lot of duplicate code to maintain. The telemetry collectors are not always very simple code (e.g. see the Lens telemetry). We would need to basically have all that duplicated for the dashboard embedded Lens visualizations. This will create a lot of maintainence burden.

Generic system to extend collectors on dashboards

This means we would need to write a system for dashboards that different embeddable implementations can plugin their own telemetry collectors. This would basically then get the embedded states and can gather telemetry data from them.

Advantages:

• Each plugin would still be in control of their own telemetry.
• Dashboard does not need to have internal knowledge about the internals of other plugin's saved object structure.
• Potentially less duplicate code to maintain, since the applications could use the same implementation for dashboard and saved object telemetry.

Disadvantages:

• A significantly more complex implementation needed (that also needs to be maintained).
• We cannot collect the telemetry data necessarily the same way, than we can on saved objects (e.g. Lens is doing aggregations on the index directly based on some fields, we cannot do that, since we're not having the state in dashboard in an aggregetable way).

This is in my opinion a larger task we still need to discuss and I'd like to involve @stacey-gammon into this.

cc @timductive @flash1293

[AlonaNadler]

Once visualizations are embedded in Dashboards directly we'll lose the existing Telemetry on them.

Which telemetry will we lose? will it be all visualizations counters? Lens? or the dashboard counters?

[timroes]

Once visualizations are embedded in Dashboards directly we'll lose the existing Telemetry on them.

Which telemetry will we lose? will it be all visualizations counters? Lens? or the dashboard counters?

I'm sorry. I phrased this very poorly.

For visualizations embedded directly in dashboards no further telemetry would be collected (none of the current visualizations or lens telemetry). The existing telemetry will still be there, and for visualizations created in the library this telemetry will be collected as it was so far. But visualizations embedded inside a dashboard are "invisible" to those telemetry counters. That's why we need to solve this problem in one of the above ways, before merging that effort (or being okay without that telemetry, which I don't think we are).

[timroes]

We synced around that topic with the App Arch, Stacey and the telemetry team. Here is the summary:

• We will need to build that generic system to collect that information, since none of the other solutions will either provide us telemetry or will work technically (e.g. because of having cross-links between OSS code and Basic code).
• We discussed several ways how that system could work (living inside the embeddable layer, using embeddable meta informations, being a general helper in the telemetry system). So far it seems we need a generic utility for that (potentially part of Persistable State Service), that allows collecting telemetry information from any part of state, that could be embedded inside another state. Embeddables should most likely then provide a getTelemetryData interface that can be used to look up their data. Somehow all of this needs to be collected and stored.
• App Arch will be responsible for the design of the service (since it goes alongside the Persistable State Service). @ppisljar will come up with some RFC/draft/start of discussion around that service soonish.