Skip to content

Record model thinking/reasoning #168

@dannykopping

Description

@dannykopping

OpenAI and Anthropic's models may respond with "thinking" or "reasoning" blocks before a tool call is determined. Capturing these "thoughts" may provide auditors with key information about why a particular tool was called.

Here is some research about how these providers support this feature.

This will introduce a new recording primitive:

type ModelThoughtRecord struct {
	InterceptionID string
	ToolUsageID    string // This is an optional relation to a `ToolUsageRecord`.
	Content        string
	Metadata       Metadata
	CreatedAt      time.Time
}

We only care about these thoughts when a tool call is to be invoked. We don't capture regular inference output, and these thoughts are just a subset of that.

Acceptance criteria:

  • Model thoughts/reasoning is recorded by all interceptor implementations
    • Messages
      • blocking
      • streaming
    • Chat Completions
      • blocking
      • streaming
    • Responses
      • blocking
      • streaming
  • Recorder types added
  • Tests added

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions