Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/attributes] Add support for profiles signal #35981

Open
mx-psi opened this issue Oct 24, 2024 · 15 comments
Open

[processor/attributes] Add support for profiles signal #35981

mx-psi opened this issue Oct 24, 2024 · 15 comments
Labels
data:profiles Profiles related issues internal/filter on hold This is blocked by another PR/issue priority:p2 Medium processor/attributes Attributes processor

Comments

@mx-psi
Copy link
Member

mx-psi commented Oct 24, 2024

Component(s)

processor/attributes

Describe the issue you're reporting

This processor would be useful for testing and validation of the experimental profiling signal.

As inspiration for how to add support this PR can be used: open-telemetry/opentelemetry-collector/pull/11071 (there is no processor with support as of the writing of this issue)

Example of configuration using profiles (needs service.profilesSupport feature gate to be enabled):

receivers:
  otlp:
    protocols:
      grpc:
exporters:
  otlp:
    endpoint: ${OTLP_ENDPOINT}
service:
  pipelines:
    profiles:
      receivers: [otlp]
      exporters: [otlp]
@mx-psi mx-psi added help wanted Extra attention is needed priority:p2 Medium processor/attributes Attributes processor data:profiles Profiles related issues labels Oct 24, 2024
Copy link
Contributor

Pinging code owners for processor/attributes: @boostchicken. See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@odubajDT
Copy link
Contributor

Hi, if possible, I would like to look at this issue

@mx-psi
Copy link
Member Author

mx-psi commented Oct 28, 2024

@odubajDT Sure, go ahead! Profiles implementation may be a bit more complex since there are multiple places where there can be attributes. Feel free to ping me once you have the PR so we can find the right reviewers

@odubajDT
Copy link
Contributor

odubajDT commented Oct 31, 2024

After a brief investigation, there is a need to add support for profiles in pkg/ottl, therefore this issue depends on #36104

Additionally, support in internal/filter needs to be added as well, I will start working on this as part of my PR, but same for the processor, there is a dependency on pkg/ottl profiles support

@odubajDT
Copy link
Contributor

/label internal/filter

Copy link
Contributor

Pinging code owners for internal/filter: @open-telemetry/collector-approvers. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@TylerHelmuth
Copy link
Member

I want to throw out the idea that we don't add profile support to any component listed in #18643 and instead rely on profiles support in the transform processor.

@mx-psi
Copy link
Member Author

mx-psi commented Oct 31, 2024

I am okay with that, however there are changes coming to the profiles data model that may mean churn in the OTTL implementation. @dmathieu and the Profiling SIG can help inform what OTTL functions are 'safer' (in that we don't expect that part of the data model to change)

@dmathieu
Copy link
Member

dmathieu commented Nov 4, 2024

Seeing the number of changes currently being made in the OTLP for profiles, I think we can assume there won't be any "safe" location.

@mx-psi
Copy link
Member Author

mx-psi commented Nov 4, 2024

Then up to @TylerHelmuth to decide if they are willing to deal with the churn. We may also not do this and not do OTTL for now, focusing only on components that deal with the resource only/deal with the proto in an abstract way

@TylerHelmuth
Copy link
Member

If the proto for profiles is churning, then pdata is churning, which means any attributes and OTTL will churn. It sounds like we should wait.

@dmathieu
Copy link
Member

dmathieu commented Nov 6, 2024

I agree to wait until the currently planned changes are shipped, after which I hope the profo should stabilize a little bit.
We shouldn't wait for them to be stable though, as that would prevent folks from experimenting with profiles, and make said stabilization rather harder.

@mx-psi mx-psi added on hold This is blocked by another PR/issue and removed help wanted Extra attention is needed labels Nov 6, 2024
@mx-psi
Copy link
Member Author

mx-psi commented Nov 6, 2024

Okay, let me put this on hold for now. @odubajDT see above, @dmathieu and the Profiling SIG will keep us updated when this can be marked as ready for review

@odubajDT odubajDT removed their assignment Nov 6, 2024
@TylerHelmuth
Copy link
Member

When we are ready to come back to this I do think we should only add support to the transformprocessor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data:profiles Profiles related issues internal/filter on hold This is blocked by another PR/issue priority:p2 Medium processor/attributes Attributes processor
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants