Skip to content

Conversation

@GeLi2001
Copy link
Contributor

@GeLi2001 GeLi2001 commented Oct 22, 2025

resolves #2194
Screenshot 2025-10-28 at 10 30 47 AM


Note

Adds proper handling of Part.inline_data and Part.file_data (images/files) with image semconv, Phoenix-friendly input serialization, plus new tests, example, and test dependency updates.

  • Instrumentation (Google GenAI):
    • Handle Part.inline_data and Part.file_data:
      • Images: emit message.content.image.image_url with data URLs or file URIs using OpenInference image semantics.
      • Non-images (e.g., PDF): record descriptive [File: mime, size/uri] in message.content.
    • Add Phoenix-friendly input serialization replacing binary data with data URLs/descriptions; integrate into _as_input_attributes.
    • Preserve image attributes during part flattening; extract tools as high-priority; downgrade unknown part handling to debug.
  • Examples:
    • New examples/generate_content_with_images.py demonstrating inline/URI images, PDFs, and async usage.
  • Tests:
    • Add coverage for inline_data/file_data handling and ensure existing text/function parts remain unchanged.
  • Dependencies:
    • Update test extras: opentelemetry-sdk>=1.38.0, opentelemetry-exporter-otlp>=1.38.0.

Written by Cursor Bugbot for commit fd65407. This will update automatically on new commits. Configure here.

@GeLi2001 GeLi2001 requested a review from a team as a code owner October 22, 2025 21:18
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. labels Oct 22, 2025
cursor[bot]

This comment was marked as outdated.


def _flatten_parts(self, parts: list[Part]) -> Iterator[Tuple[str, AttributeValue]]:
content_values = []
tool_call_index = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will

 yield (
                    f"{MessageContentAttributes.MESSAGE_CONTENT_IMAGE}.{ImageAttributes.IMAGE_URL}",
                    data_url,
                )

get ignored or inadvertently converted into a singular message_content in _flatten_parts? Seems like we only check for certain attributes, otherwise we flatten into a single message. Don't we want to preserve this image_url attribute?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yess thanks for bringing this up, I have reviewed and updated the pr so that the attributes are exported and rendered correctly on phoenix as shown in latest screenshot above in description

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@cursor
Copy link

cursor bot commented Oct 23, 2025

Bug: Image Overwrite Due to Missing Indexing

Multiple image parts within a single message overwrite each other in the trace attributes. This happens because the generated OpenInference attributes for images lack indexing, causing all images to use the same key and only the last image's data to be preserved.

Fix in Cursor Fix in Web

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Oct 28, 2025
logger.debug(f"Non-text part encountered: {part}")
else:
# TODO: Handle other types of parts
logger.debug(f"Non-text part encountered: {part}")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Incorrect Indentation Causes Debug Log Misplacement

The else clause is incorrectly indented, associating it with the inner for loop instead of the if/elif chain. This causes the debug log "Non-text part encountered" to print after processing all attributes for each part, even when they're handled correctly. This else clause is also redundant, as unsupported part types are now handled in _get_attributes_from_part.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[bug] "Other field types of parts are not supported yet" error when attaching images in GoogleGenAIInstrumentor

3 participants