feat: Enhanced Logging System with Error Tracking and Debug Integration #50
Description
Summary
Enhance the existing logging functionality to address structural issues, implement comprehensive error tracking, and integrate advanced debugging capabilities using the debug npm package.
Current Issues Identified
1. Deep Nested Log Folder Structure
Current logging structure shows unnecessary nesting:
./undefined/.logs/.logs./comm/.logs/.logs./test/logs/.logs
This creates confusing directory hierarchies and potential path resolution issues.
2. Missing Centralized Error Logging
Currently, errors from MCP server responses and tool failures are not systematically captured in a dedicated error log for analysis and training purposes.
Proposed Enhancements
1. Fix Log Directory Structure
- Problem: Unnecessary nested
.logs/.logsdirectories - Solution: Refactor EventLogger.ts constructor logic to prevent path duplication
- Impact: Cleaner file system structure and improved path resolution
2. Enhanced Error Logging System (error.log)
Create a dedicated error logging system that captures:
Error Sources to Track:
- ✅ MCP Server Response Errors: Any error responses from the MCP server
- ✅ Tool Execution Failures: Tools that end with non-zero exit codes
- ✅ Runtime Exceptions: Unhandled errors in core operations
- ✅ Validation Failures: Input validation and constraint violations
- ✅ Network/Connection Issues: Communication failures with external services
Error Log Format:
interface ErrorLogEntry {
timestamp: Date;
source: 'mcp_server' | 'tool_execution' | 'runtime' | 'validation' | 'network';
operation: string;
agent: string;
taskId?: string;
error: {
message: string;
name: string;
code?: string | number;
stack?: string;
};
context: {
tool?: string;
parameters?: Record<string, unknown>;
responseCode?: number;
retryCount?: number;
};
severity: 'low' | 'medium' | 'high' | 'critical';
metadata?: Record<string, unknown>;
}Integration Points:
- ResponseEnhancer: Use error patterns to improve response quality
- AccountabilityTracker: Track error attribution and resolution
- Agent Training: Provide error pattern data for model improvement
- Workflow Enhancement: Identify bottlenecks and failure modes
3. Debug Package Integration
Integrate the debug npm package (4.4.3) for enhanced debugging capabilities across all /src code.
Debug Package Capabilities:
- Namespace-based filtering:
agent-comm:core:*,agent-comm:tools:* - Environment control:
DEBUG=agent-comm:*orDEBUG=*,-agent-comm:network - Color-coded output: Visual distinction between components
- Timing information: Millisecond differences between operations
- Custom formatters: Structured object inspection
- Dynamic control: Runtime enable/disable of debug categories
Proposed Debug Namespace Structure:
agent-comm:core:accountability # AccountabilityTracker operations
agent-comm:core:compliance # ComplianceTracker operations
agent-comm:core:connection # ConnectionManager operations
agent-comm:core:delegation # DelegationTracker operations
agent-comm:core:response # ResponseEnhancer operations
agent-comm:core:context # TaskContextManager operations
agent-comm:tools:create-task # Task creation operations
agent-comm:tools:archive # Archival operations
agent-comm:tools:progress # Progress tracking
agent-comm:logging:event # EventLogger operations
agent-comm:logging:error # Error logging operations
agent-comm:resources:* # Resource provider operations
agent-comm:prompts:* # Prompt management operations
agent-comm:utils:* # Utility function debugging
Implementation Strategy:
npm install debug @types/debug- Create debug instances in each major component
- Replace or supplement existing console.log statements
- Add performance timing for critical operations
- Implement structured object logging for complex data
Technical Implementation Plan
Phase 1: Fix Log Directory Structure
- Audit current logging paths in EventLogger.ts constructor
- Fix path resolution logic to prevent duplication
- Migrate existing log files to corrected structure
- Update tests to reflect new directory structure
Phase 2: Enhanced Error Logging
- Create ErrorLogger class extending EventLogger functionality
- Implement error capture middleware for MCP responses
- Add error tracking to tool execution pipeline
- Create error analysis utilities for pattern detection
- Integrate with ResponseEnhancer and AccountabilityTracker
Phase 3: Debug Package Integration
- Install debug package and types
- Create debug namespace hierarchy across all /src modules
- Replace existing logging statements with debug calls
- Add performance timing instrumentation
- Document debug usage patterns for development
Phase 4: Integration & Training Features
- Error pattern analysis tools for agent training data
- ResponseEnhancer integration with error history
- AccountabilityTracker error attribution
- Workflow optimization based on error patterns
Acceptance Criteria
Log Directory Structure
- No nested
.logs/.logsdirectories exist - All logs write to single
.logsdirectory per component - Existing log files migrate successfully
- Tests pass with new directory structure
Error Logging System
- All MCP server errors captured in error.log
- Tool execution failures logged with context
- Error severity classification working
- Error log rotation and archival implemented
- Integration with ResponseEnhancer functional
- Integration with AccountabilityTracker functional
Debug Package Integration
- Debug namespaces implemented across all /src code
- Environment variable filtering working (
DEBUG=agent-comm:*) - Color-coded output functioning in terminal
- Performance timing added to critical operations
- Documentation updated with debug usage examples
Training & Analysis Features
- Error pattern analysis tools implemented
- Agent training data export functionality
- Workflow enhancement recommendations system
- Error resolution tracking and metrics
Benefits
- Improved Debugging Experience: Granular control over debug output
- Enhanced Error Visibility: Systematic error capture and analysis
- Better Agent Training: Rich error data for model improvement
- Workflow Optimization: Data-driven process improvements
- Cleaner File System: Resolved directory structure issues
- Better Maintainability: Structured logging across all components
Dependencies
debug@^4.4.3- Core debugging functionality@types/debug- TypeScript definitions- Existing EventLogger functionality
- Current MCP server infrastructure
Testing Strategy
- Unit tests for ErrorLogger class
- Integration tests for debug namespace functionality
- End-to-end tests for error capture pipeline
- Performance tests for logging overhead
- Migration tests for existing log data
Priority: High - Addresses critical infrastructure improvements and enables advanced debugging capabilities for better development and operational insights.