[O11y][AWS Billing] Lens Migration

[harnish-crest-data]

Manually migrate AWS Billing visualizations to the lens in the current Kibana version 8.8.1 itself.

Preparation of data for testing

• Use sample_event.json / setup live instance using docker / mock data using a mock server to populate dashboards

Migration stats

AWS Billing Dashboards	Before Migration				After Migration
	Maps	Search	Lens	Visualization	Maps	Search	Lens	Visualization
[Metrics AWS] Billing Overview	0	0	0	4	0	0	4	0

Dashboard : [AWS Billing] Dashboards

• Migrate 4 visualization to the lens.
  • [O11y][AWS] Migrate AWS Billing dashboard visualizations to lens #6644

Verification and Validation

• Verification of functionality remains the same after migration in Kibana
• Verification of data count of after-migrated visualization with before-migrated visualization in Kibana
• Verification of Dashboards is not distorted in the supported Kibana version (TBD)

[harnish-crest-data]

Hi @drewdaemon / @kaiyan-sheng,

While migrating the Billing dashboard metric visualization, found that the counts are not getting matched for some cases.

Please refer the below before migrated screenshot and count shown in the metric chart,

Now with the same time range, please refer the after migrated screenshot and count shown in the metric chart,

While migrating into lens, we have used reduced timerange to 12h. While updating the date and time ranges into exact 12h or 12h multiple formats, the counts are getting matched
Please refer to the below screenshots for more understanding!

Before migrated:

After migrated:

But there could be a potential issue related to the reduced timerange buckets. If you have defined the reduced time range to 12h, it will create the 12h buckets and from all those buckets, it will show the last value of the bucket. Now if the date and time range is not in multiple of the defined reduced time range (In our case 12h), it will create the partial bucket (For example, If the defined time range is 23rd May 12:00 pm to 24th May 3:00 pm then it will create 3 buckets. 2 buckets with 12h data and 1 bucket with the rest 3h data! So the data with 3h is a partial bucket in this case.) and show the value of that partial bucket. While in the TSVB if the bucket value is partial then it won't be considered as last_bucket. Hence the counts are getting mismatched in those cases. Correct me if I understood wrongly.

Also from the visual perspective, Please refer to the below screenshots.

Before migration:

Now by clicking on the edit in lens option, it will migrate this visualization to metric type, in which we are not able to prefix the value with $ sign. Also, the values are automatically converted into the k suffix. We are not able to stop this converter. Please refer to the below screenshot.

If we need to match the lens visualization, before visualization we need to convert it into legacy metrics. In which we have the suffix options and values are looks similar as well as more readable as per user perspective. Please refer to the below screenshot.

Let me know how we can proceed with this metric chart!

[drewdaemon]

Different numbers problem

I'm assuming that the TSVB visualization had Data timerange mode set to Last value with an Auto interval?

Lens reduced time range simply adds a filter to the query so that it only includes results from whatever interval you set (the last 12 hours in your case, the last 30 seconds in this screenshot).

I would expect the values to be different from TSVB since they do somewhat different things.

I'm curious, is this metric a gauge or a counter?

If it's a counter, we can simply use Lens's Last value function to get the value from the last document.

If it's a gauge value, I'm curious what the user should really be seeing? Maybe that's a question for @kaiyan-sheng .

Currency formatting problem

To properly format the number in Lens, consider removing the $ suffix and setting the format of the data view field instead (you can create an ad-hoc data view if you don't want the format to apply everywhere).

Screen.Recording.2023-06-02.at.2.34.02.PM.mov

[kaiyan-sheng]

@drewdaemon Sorry for the late response! EstimatedCharges is an interesting metric. I would say it's gauge because this number clears to zero every month. But in each month, it behaves like a counter. In our dashboard, I'd like to show the last value we collected from CloudWatch for EstimatedCharges just to give users an idea of what would be the estimated bill looks like when they open this dashboard. I think Last value would be appropriate for this case.

[drewdaemon]

@harnish-elastic all the questions I can see have been answered so you should be unblocked. Let us know if you need further guidance.