Azure Event Hub receiver Semantic Convention translator #32486
Conversation
|
I like this idea and I think it makes a lot of sense to try to map to Semantic Conventions were possible. Can you make sure an Issue is created to track this against? |
|
Can you please also contribute those transformations to https://github.com/open-telemetry/semantic-conventions so we have it in the spec? |
|
Going along with @atoulme's request, I think it may be good to block this PR until the suggested conventions here have been approved and merged into the semantic conventions repo, as any changes requested there will need to be reflected in this package (and PR). The problem with going forward with this PR before semantic conventions are finalized, would be the decisions there could end up being breaking changes here (if this gets merged and released). Let me know if I've misunderstood the nature of this change though 👍 |
|
@atoulme @crobert-1 @cparkins I will get a semconv issue/PR opened to add the net-new attribute names. As for the mapping of azure fields to existing fields, I do not believe the semantic convention currently defines those. The closest I know about is are these appendices . I will open a issue/PR for that as well. |
|
This is still a WIP. The work to add semmantic conventions for the new azure namespace is on going. |
|
Is it worth converting this PR to draft until it is ready for review? |
|
Yep, I think that's wise given it's blocked on semantic conventions PR: |
|
Note to add opt-in for identity field as it contains PII data. |
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
We still want this PR to land, it's just waylaid by semconv work. The current direction looks like we'll move this translator towards using a defined event structure instead of attributes but not confirmed. Is there a label we can apply to indicate this isn't stale but work in progress? |
|
There isn't a good label for that, but I can add |
crobert-1
added
the
never stale
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
As an update, the semconv PR has picked up again and we'll be moving to an event model instead of a generic log. Named events have the benefit of a clear schema of field names where logs do not. Hopefully that PR will wrap up soon and we can update this work to reflect the changes. |
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
This PR makes the `never stale` label respected on PRs. I noticed on #32486 that I had added the `never stale` label, but `Stale` was still added. This is because we need to set the [`exempt-pr-labels` option](https://github.com/actions/stale?tab=readme-ov-file#exempt-pr-labels) in the [stale action](https://github.com/actions/stale) workflow.
**Description**: This PR adds a new Azure Resource Logs translator that can be used to convert Azure events into resource logs events using Semantic conventions defined here [here](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/azure/events.md). This supersedes following draft PR by updating it to have the latest semconv: - #32486 *Note*: A follow-up PR will update the Azure receiver to use the new translator. **Testing**: Includes unit tests to verify expected behaviours and data structures. **Documentation**: --- cc @markrendle @lmolkova @TylerHelmuth --------- Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Mark Rendle <mark@rendlelabs.com> Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Curtis Robert <crobert@splunk.com> Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
|
This PR can be closed now #34830 has merged. |
👍🏻 |
**Description**: This PR adds a new Azure Resource Logs translator that can be used to convert Azure events into resource logs events using Semantic conventions defined here [here](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/azure/events.md). This supersedes following draft PR by updating it to have the latest semconv: - open-telemetry#32486 *Note*: A follow-up PR will update the Azure receiver to use the new translator. **Testing**: Includes unit tests to verify expected behaviours and data structures. **Documentation**: --- cc @markrendle @lmolkova @TylerHelmuth --------- Signed-off-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Mark Rendle <mark@rendlelabs.com> Co-authored-by: Alex Boten <223565+codeboten@users.noreply.github.com> Co-authored-by: Curtis Robert <crobert@splunk.com> Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
Description:
This PR changes the translator used by the Azure Event Hubs receiver to change property names on log records and metrics to the equivalent OTel Semantic Conventions names, making it easier to query logs and metrics ingested in this way alongside other logs and metrics.
Testing: WIP.
Documentation: WIP