Skip to content

Improving pre-aggregated Standard Metrics in SDK #1233

Closed

Description

Background read: https://docs.microsoft.com/en-us/azure/azure-monitor/app/pre-aggregated-metrics-log-metrics

The UI screenshots in this post are from Metric Explorer.

The default configuration of SDK (when used with Web, AspNetCore or WorkerService nugets) adds a special telemetry processor called AutocollectedMetricsExtractor before sampling processors. This processor inspects every Request and Dependency telemetry items, and aggregates them with key dimension. This pre-agg metric is sent to Application Insights once every min, and are available to query under StandardMetrics->Server in metric explorer.

As this metric is calculated before sampling, these metrics are superior in terms of accuracy.
There are 5 standard metrics which are populated from the above.
image

They are:
Dependency Calls - Count of dep calls.
Dependency duration - Duration of dep calls
Server request - Count of incoming Server request
Server response time - Duration of Server requests
Server request rate - Derived metric from server request count showing requests/sec.

As explained in the docs, these counters are sent with certain key dimensions only. (instead of entire dimensions available in log-based metrics). In UI, these dimensions allow splitting the metric chart, and Alerting allows alerting based on these dimensions.

For Requests, currently the only dimension populated by SDK is 'successful request', even though the UI lists 6 dimensions.
image

For Dependencies, SDK populates dimensions Dependency type and Sucessful call, though UI lists 6 dimensions.
image

As SDK pre-aggregation did not add all the dimensions supported in UI, it was not possible to split the charts using these dimensions. (eg: it is not possible to split based on cloudRoleInstance)

We plan to improve the metric experience for the Standard Metrics, by adding more dimension to the SDK pre-aggregators.

This will be done in several steps:
1 Add dimensions which are low/fixed cardinality like CloudRoleName, CloudRoleInstance, Dependency/Request performance, real/synthetic. Users will be allowed to pick a max limit to the number of values for these. (similar to MaxDependencyTypesToDiscover)

  1. Add dimensions like OperationName, DependencyName which can potentially be high cardinality. We'll provide option for user to configure this. For example, user could say configure a regex on the OperationName before applying it as a dimension. (these needs to designed)

  2. Allow users to add any custom dimensions. (this also need to be designed)

#1 is planned to be addressed soon (in 2.12.0 version of SDK (to be released as stable by end of year, with betas before)
#2, #3 will follow later. (no ETAs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions