Skip to content

Comments

feat: enhance decision tracking with tool result visibility#23

Merged
douglaz merged 6 commits intomasterfrom
feat/decision-tracking-tool-visibility
Aug 31, 2025
Merged

feat: enhance decision tracking with tool result visibility#23
douglaz merged 6 commits intomasterfrom
feat/decision-tracking-tool-visibility

Conversation

@douglaz
Copy link
Owner

@douglaz douglaz commented Aug 31, 2025

Summary

  • Implements comprehensive decision tracking with unique IDs throughout the agent's execution flow
  • Makes tool execution results visible in agent memory so the agent can observe actual content
  • Replaces non-deterministic time-based filtering with count-based approach for consistent behavior

Changes

  • Added decision tracking structures: DecisionRecord and DecisionResult for structured decision tracking
  • Implemented decision ID flow: Records decisions with unique IDs and updates them with results after execution
  • Tool result visibility: Stores tool execution results in memory with actual content (HTTP responses, file contents, etc.)
  • Count-based filtering: Shows last 3 tool results instead of time-based filtering for deterministic behavior
  • Content truncation: Prevents context explosion by truncating large results to ~1000 characters
  • Comprehensive tests: Added 8 new test cases covering all new functionality

Technical Details

The agent can now see what tools actually returned rather than just knowing they were executed. For example:

  • HTTP GET requests: The agent sees the actual HTML/JSON response
  • File operations: The agent sees file contents that were read
  • Command execution: The agent sees command output

The count-based filtering (last 3 results) ensures:

  • Fast agents don't accumulate too many results
  • Slow agents still see their results
  • Deterministic behavior across different execution speeds

Test Plan

  • Unit tests for DecisionRecord/DecisionResult structures
  • Tests for decision ID tracking and updates
  • Tests for tool result storage and visibility
  • Tests for count-based filtering
  • Tests for content truncation
  • Tests for memory statistics accuracy
  • All 38 tests passing
  • Clippy warnings resolved
  • Code formatted with cargo fmt

- Add DecisionRecord and DecisionResult structures for tracking agent decisions
- Implement decision ID tracking throughout execution flow
- Make tool execution results visible in agent memory
- Replace time-based filtering with deterministic count-based approach (last 3 results)
- Add content truncation to prevent context explosion
- Include comprehensive unit tests for all new functionality

This allows the agent to see the actual content returned by tools (HTTP responses,
file contents, etc.) rather than just knowing that tools were executed. The
count-based filtering ensures deterministic behavior regardless of agent speed.
@douglaz douglaz force-pushed the feat/decision-tracking-tool-visibility branch from 14101c0 to 80fd00e Compare August 31, 2025 20:07
- Add get_tools_with_schemas method to MCP client for tool discovery
- Add timeout_secs to LLMConfig for configurable timeouts
- Implement smart timeout defaults for Ollama based on model size (30min for 70b/405b, 5min for others)
- Update tests to include timeout_secs field
@douglaz douglaz force-pushed the feat/decision-tracking-tool-visibility branch from 80fd00e to e724c7f Compare August 31, 2025 20:08
- Replace positional format arguments with named placeholders
- Use direct variable names in placeholders where applicable
- Use named parameters only for methods and properties
- Ensure all format strings follow CONVENTIONS.md guidelines
- Fix redundant format arguments caught by clippy
Simplify timeout logic to avoid model-specific checks that may not
cover all cases. A 30-minute timeout is reasonable for all models.
Use named parameters for string interpolation as per CONVENTIONS.md
@douglaz douglaz merged commit 9c6caba into master Aug 31, 2025
4 checks passed
@douglaz douglaz deleted the feat/decision-tracking-tool-visibility branch August 31, 2025 20:56
douglaz added a commit that referenced this pull request Sep 1, 2025
- Implement intelligent JSON-aware truncation that preserves structure
- Add error handling for failed decision parsing (prevents NULL results)
- Improve parameter serialization using JSON instead of Debug format
- Include complete tool output for all action types (Remember, Wait, Explore)
- Remove redundant tool_result_* memory entries (outputs only in decisions)
- Add metadata fields (_truncated, _original_size) for truncated outputs
- Recursively handle nested objects in truncation algorithm

These improvements build on PR #23's foundation by adding:
1. Smart truncation vs basic string cutting
2. Error result handling for invalid actions
3. Better parameter display format
4. Complete output tracking for all actions
douglaz added a commit that referenced this pull request Sep 1, 2025
)

- Implement intelligent JSON-aware truncation that preserves structure
- Add error handling for failed decision parsing (prevents NULL results)
- Improve parameter serialization using JSON instead of Debug format
- Include complete tool output for all action types (Remember, Wait, Explore)
- Remove redundant tool_result_* memory entries (outputs only in decisions)
- Add metadata fields (_truncated, _original_size) for truncated outputs
- Recursively handle nested objects in truncation algorithm

These improvements build on PR #23's foundation by adding:
1. Smart truncation vs basic string cutting
2. Error result handling for invalid actions
3. Better parameter display format
4. Complete output tracking for all actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant