Skip to content

Conversation

@keith-decker
Copy link
Contributor

@keith-decker keith-decker commented Sep 4, 2025

Description

The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI.
This PR provides APIs to minimize the work needed to instrument genai libraries. It provides helpers for starting and stopping LLM invocations, it generates spans aligned with semconvs.

Metrics and Events to come on future PRs

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Basic unit test for starting and stopping LLM - Spans
  • Unit test for parent/child invocations

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@keith-decker keith-decker requested a review from a team as a code owner September 4, 2025 15:33
from typing import List, Literal, Optional, Type, TypedDict


class TextPart(TypedDict):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very similar to #3718 @DylanRussell

@DylanRussell
Copy link
Contributor

Would it be possible to break the PR up a bit ? It's kind of large. Maybe we can agree on the types in 1 PR. Even still it's a bit large

@linux-foundation-easycla
Copy link

CLA Missing ID CLA Not Signed

@keith-decker
Copy link
Contributor Author

Creating a new PR - a merge in our working repo polluted this git history.

#3768

@keith-decker keith-decker deleted the util-genai-inference branch October 13, 2025 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants