Skip to content

Conversation

@ultmaster
Copy link
Contributor

@ultmaster ultmaster commented Aug 12, 2025

Summary

  • add langsmith and langfuse tracing integrations
  • extend tracing backend selection and logging in API

Testing

  • npm run build-webview
  • npm run build-cli
  • npm run lint
  • npm test (failed: hung at compile step)
  • python -m pytest python/tests

https://chatgpt.com/codex/tasks/task_e_6899af15e0b0832e8c558f1fa495399d

@ultmaster ultmaster changed the title feat: add langsmith and langfuse integration feat: add langfuse integration Aug 14, 2025
@ultmaster ultmaster marked this pull request as ready for review August 14, 2025 10:54
Copilot AI review requested due to automatic review settings August 14, 2025 10:54
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds Langfuse integration to the tracing system, allowing users to log POML calls to Langfuse alongside existing backends (Weave, AgentOps, MLflow). The integration follows the established pattern of requiring local tracing to be enabled first.

  • Add Langfuse as a new tracing backend option
  • Implement Langfuse logging integration with prompt creation and observation tracking
  • Include manual test examples for both direct Langfuse usage and POML integration

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
python/poml/api.py Adds Langfuse backend support and refactors result handling for consistency
python/poml/integration/langfuse.py Implements Langfuse-specific logging functionality
python/tests/manual/example_langfuse_poml.py Manual test demonstrating POML with Langfuse tracing
python/tests/manual/example_langfuse_original.py Manual test showing direct Langfuse usage

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

enabled = [enabled]

global _trace_enabled, _trace_dir, _weave_enabled, _agentops_enabled, _mlflow_enabled
global _trace_enabled, _trace_dir, _weave_enabled, _agentops_enabled, _mlflow_enabled, _langsmith_enabled, _langfuse_enabled
Copy link

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable _langsmith_enabled is referenced in the global declaration but is not defined anywhere in the visible code. This will cause a NameError when the function is called.

Suggested change
global _trace_enabled, _trace_dir, _weave_enabled, _agentops_enabled, _mlflow_enabled, _langsmith_enabled, _langfuse_enabled
global _trace_enabled, _trace_dir, _weave_enabled, _agentops_enabled, _mlflow_enabled, _langfuse_enabled

Copilot uses AI. Check for mistakes.
type="text",
prompt=prompt
)

Copy link

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable prompt_client is used before it's defined. It's defined on line 18 but used on line 15, which will cause a NameError.

Suggested change
prompt_client = client.create_prompt(
name=name,
type="text",
prompt=prompt
)
@observe(name=name)
def poml(prompt, context, stylesheet):
client.update_current_generation(prompt=prompt_client)
return result

Copilot uses AI. Check for mistakes.
prompt=prompt
)

poml(prompt=prompt, context=context, stylesheet=stylesheet)
Copy link

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function does not return the result of the poml call, but the function signature suggests it should return Any. Add return before the function call.

Suggested change
poml(prompt=prompt, context=context, stylesheet=stylesheet)
return poml(prompt=prompt, context=context, stylesheet=stylesheet)

Copilot uses AI. Check for mistakes.

if trace_record is not None:
trace_record["result"] = result
trace_record["result"] = result_dict
Copy link

Copilot AI Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable result_dict may not be defined in all code paths. When format is "raw", result_dict is never assigned, but result is used directly. This should be trace_record["result"] = result to maintain consistency.

Suggested change
trace_record["result"] = result_dict
trace_record["result"] = result if format == "raw" else result_dict

Copilot uses AI. Check for mistakes.
@ultmaster ultmaster added this to the v0.1 milestone Aug 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants