Skip to content

Missing Human-in-the-Loop Tracing Support #401

Description

@prajwalun

Title: Missing tracing support for human-in-the-loop agent workflows

Description:
My resume agent pauses for human input when it detects potential fabrication risks or needs clarification. The current tracing API doesn't handle these pause/resume scenarios properly, making it difficult to track the complete workflow.

Problem:
When my agent pauses for user clarification, the traces break or become disconnected:

if fabrication_risk['needs_clarification']:
    # How do we trace this pause for user input?
    request_clarification()
    continue  # Resume processing after user input

This creates fragmented traces that are hard to debug and monitor.

Expected Behavior:
Tracing should support pause/resume patterns for human-in-the-loop workflows:

# Option 1: Pause/resume context managers
with judgment.pause_for_human_input(reason="clarification_needed"):
    user_response = await get_user_clarification()
    
with judgment.resume_from_human_input(previous_span_id=span_id):
    continue_processing(user_response)

# Option 2: Session continuation
session_id = judgment.get_current_session_id()
# ... pause for user input ...
judgment.resume_session(session_id, user_input=user_response)

Current Behavior:

  • Traces become fragmented when agent execution pauses for human input
  • No way to correlate spans before and after human interaction
  • Difficult to debug and monitor complete workflows
  • Session continuity is lost

Impact:
This makes it impossible to properly monitor and debug human-in-the-loop agent workflows, which are common in production systems.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions