-
Notifications
You must be signed in to change notification settings - Fork 1
refactor: simplify callback inputs and add telemetry support #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Simplify callback inputs to use raw args/kwargs instead of inspecting function parameters - Add provider field to OpenAILM for telemetry tracking - Fix streaming parser to handle empty choices gracefully - Improve adapter output validation logic for clarity These changes make it easier to integrate external telemetry libraries like PostHog. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Update test_module_callbacks to expect new simplified callback input structure with raw args/kwargs instead of nested inputs dict. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR simplifies the callback system to support external telemetry integrations by passing raw function arguments instead of mapped parameters, adds provider tracking to OpenAILM for telemetry purposes, and includes defensive improvements to streaming response handling.
Key Changes
- Callback system now passes raw
argsandkwargsinstead of usinginspect.getcallargs()to map parameter names, making it simpler and more flexible for telemetry libraries - Added
providerfield to OpenAILM class to track which LM provider is being used (e.g., "openai", "bedrock", "ollama") - Improved streaming robustness with empty choices guard and clearer output validation logic
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/udspy/callback.py | Simplified callback inputs to pass raw args and kwargs instead of inspecting function signatures |
| src/udspy/lm/openai.py | Added provider parameter to OpenAILM for telemetry tracking |
| src/udspy/lm/factory.py | Updated LM factory to pass provider parameter when creating OpenAILM instances |
| src/udspy/adapter.py | Added defensive guard for empty choices in streaming and improved output validation readability |
| tests/test_callback.py | Updated test assertions to work with new callback input structure |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Remove self/instance from inputs | ||
| inputs.pop("self", None) | ||
| inputs.pop("instance", None) | ||
| inputs = {"kwargs": kwargs, "args": args} |
Copilot
AI
Nov 19, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The inputs structure has changed from flattened key-value pairs to {"kwargs": kwargs, "args": args}, but the docstrings in the BaseCallback class (lines 64, 94, and 124) still describe inputs as "Input arguments as key-value pairs" or "LLM input parameters". The documentation should be updated to reflect this new structure.
For example, the relevant docstrings should describe the new format:
inputs: Dict containing 'kwargs' and 'args' keys with the function's arguments
Move openai imports inside methods to defer loading until needed, reducing initial import time and memory usage. Also add unreachable assertion to satisfy mypy return type checking. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Summary
Motivation
These changes make it easier to integrate external telemetry libraries like PostHog by providing simpler, more flexible callback inputs and tracking which LM provider is being used.
Changes
with_callbacksto pass rawargsandkwargsinstead of usinginspect.getcallargs()providerparameter toOpenAILMclassproviderparameter when creatingOpenAILMinstanceschoicesin streaming parserTest plan
🤖 Generated with Claude Code