Metric Exemplars SDK Specification #1828
Conversation
|
FYI I have been working on trace sampling lately, and there is a lot of alignment between that work and this work. I have made claims that we can use weighted sampling to implement a hard rate limit on spans inside a tail sampler, and by extension I claim to know how to do the same for histogram exemplars. For a histogram aggregator outputting high-resolution histogram buckets you might not want one exemplar per bucket, and with weighted sampling we can produce a fixed number (i.e., using a reservoir) of exemplars with an equal expected number of spans per bucket. This simple approach uses two passes, documented in the README of the https://github.com/lightstep/varopt repository. The more complicated approach can do this approximately in a single pass, which I also discussed briefly in today's Sampling SIG. See this draft. I will elaborate on this in a prototype exemplar reservoir tail sampler based on Varopt. |
|
@jmacd I agree I think we can do way better than status quo, especially for high-fidelity histograms. I re-tweaked the wording to leave room for an exemplar reservoir sampling specification when those land. |
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
jsuereth
added
area:sdk
jsuereth
changed the title
specification/metrics/sdk.md
Outdated
| The "offer" method SHOULD accept measurements, including: | ||
|
|
||
| - value | ||
| - `Attributes` |
specification/metrics/sdk.md
Outdated
| @@ -128,6 +128,8 @@ are the inputs: | |||
| applies to [synchronous Instruments](./api.md#synchronous-instrument). | |||
| * The `aggregation` (optional) to be used. If not provided, a default | |||
| aggregation will be applied by the SDK. The default aggregation is a TODO. | |||
| * The `exemplar_reservoir` to use for storing exemplars. | |||
There was a problem hiding this comment.
This should be marked optional right?
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: Aaron Abbott <aaronabbott@google.com>
* First cut at exemplar spec. * Update exemplar specification. * Remove the word parent Co-authored-by: Reiley Yang <reyang@microsoft.com> * Update specification/metrics/sdk.md Co-authored-by: Reiley Yang <reyang@microsoft.com> * Update specification/metrics/sdk.md Co-authored-by: Reiley Yang <reyang@microsoft.com> * Update specification/metrics/sdk.md Co-authored-by: Reiley Yang <reyang@microsoft.com> * Update specification/metrics/sdk.md Co-authored-by: Reiley Yang <reyang@microsoft.com> * Update specification/metrics/sdk.md Co-authored-by: Aaron Abbott <aaronabbott@google.com> * Fixes from review. * Clarify a confusing paragraph. Co-authored-by: Reiley Yang <reyang@microsoft.com> Co-authored-by: Aaron Abbott <aaronabbott@google.com>
Fixes #1797
Changes
Add Exemplar specification and customization to the metrics specification.
Related issues #1260 #617
Related oteps #113
Notes