[feat]:Custom Backend Tracing

[oandreeva-nv]

What does the PR do?

NOTE: currently targeting refactor PR: #7390 , after refactor is merged, will target main, or we can merge this PR to refactor and then refactor to main.

Currently Triton Trace capabilities are limited to reporting of a predefined set of activities listed here. Neither of the listed options support tracing a custom activity that a potential user might want to capture.

This PR includes frontend adjustments to support Custom Backend features, including:

• it handles new activity type TRITONSERVER_TRACE_CUSTOM_ACTIVITY
• for Triton Trace API it extracts custom activity name from trace context and reports it in the log file
• for OpenTelemetry Trace API it also extracts custom activity name from trace context
  • If custom activity name doesn't end with _START or _END, an event is added to a currently active span (i.e. on top of the stack)
  • Otherwise, it either starts or ends the currently active span
  • Docs mention that it is user's responsibility to make sure that each opened active span should be properly closed.
• Added docs
• Added tests for Triton Trace API and OpenTelemetry Trace API.

Sample OpenTelemetry trace:

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• build
• ci
• docs
• feat
• fix
• perf
• refactor
• revert
• style
• test

Related PRs:

server PR: #7403
identity backend: triton-inference-server/identity_backend#30

Where should the reviewer start?

TEP

Test plan: Added tests for Triton Trace API and OpenTelemetry Trace API.

• CI Pipeline ID: 16268312

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: Custom Backend Tracing #5777