feat: log usage tools when called by LLM

[lucasgomide]

Currently we are not firing Tool usage events when they are triggered by LLM.call

Here is an example of the log captured

==============
 Got event: {'timestamp': '2025-05-29T10:54:17.643098', 'type': 'llm_call_started', 'source_fingerprint': None, 'source_type': None, 
'fingerprint_metadata': None, 'messages': [{'role': 'user', 'content': 'What is the weather in San Francisco?'}], 'tools': [{'type': 
'function', 'function': {'name': "'get_weather'", 'description': "'Get the current weather in a given location'", 'parameters': "{'type': 
'object', 'properties': {'location': {'type': 'string', 'description': 'The city and state, e.g. San Francisco, CA'}}, 'required': 
['location']}"}}], 'callbacks': None, 'available_functions': {'get_weather': '<function create_tool_proxy.<locals>.tool_proxy at 
0x1227cb060>'}} 
==============


==============
 Got event: {'timestamp': '2025-05-29T10:54:18.540948', 'type': 'tool_usage_started', 'source_fingerprint': None, 'source_type': None, 
'fingerprint_metadata': None, 'agent_key': None, 'agent_role': None, 'tool_name': 'get_weather', 'tool_args': {'location': 'San Francisco, 
CA'}, 'tool_class': None, 'run_attempts': None, 'delegations': None, 'agent': None} 
==============


==============
 Got event: {'timestamp': '2025-05-29T10:54:18.544325', 'type': 'tool_usage_finished', 'source_fingerprint': None, 'source_type': None, 
'fingerprint_metadata': None, 'agent_key': None, 'agent_role': None, 'tool_name': 'get_weather', 'tool_args': {'location': 'San Francisco, 
CA'}, 'tool_class': None, 'run_attempts': None, 'delegations': None, 'agent': None, 'started_at': '2025-05-29T10:54:18.540925', 'finished_at':
'2025-05-29T10:54:18.544315', 'from_cache': False, 'output': "\n\nTool get_weather called successfully with parameters: {'location': 'San 
Francisco, CA'}\n\n"} 
==============


==============
 Got event: {'timestamp': '2025-05-29T10:54:18.545330', 'type': 'llm_call_completed', 'source_fingerprint': None, 'source_type': None, 
'fingerprint_metadata': None, 'response': "\n\nTool get_weather called successfully with parameters: {'location': 'San Francisco, CA'}\n\n", 
'call_type': "<LLMCallType.TOOL_CALL: 'tool_call'>"} 
==============

Here is some console screenshots

[joaomdmoura]

Disclaimer: This review was made by a crew of AI Agents.

Code Review Comment for PR #2916 - Tool Usage Logging Implementation

Overview

This pull request successfully introduces logging functionality for tool usage when called by the LLM, encompassing events for tool start, completion, and error handling. It expands the codebase to include better observability of tool interactions.

Recommendations & Improvements

Code Enhancements

1. Event Bus Assertion Handling

   • Current Implementation:

     assert hasattr(crewai_event_bus, "emit")

   • Suggested Improvement:

     if not hasattr(crewai_event_bus, "emit"):
         raise AttributeError("Event bus missing required 'emit' method")

   • Reasoning: Using assertions is not the best practice for production code. Raising an explicit error provides clearer control flow.

2. Error Message Duplication

   • Current Implementation:

     error=f"Tool execution error: {str(e)}"

   • Suggested Improvement:

     error_message = f"Tool execution error: {str(e)}"

   • Reasoning: Storing the error message in a variable avoids repetition and enhances maintainability.

3. Descriptive Variable Naming

   • Current Implementation:

     started_at = datetime.now()

   • Suggested Improvement:

     tool_execution_start_time = datetime.now()

   • Reasoning: More descriptive naming enhances code clarity.

Event Class Structure

Recommended Adjustment

• Current Fields:

  agent_key: Optional[str] = None
  agent_role: Optional[str] = None
  tool_name: str

• Suggested Organization:
  • Group required and optional fields:

  # Required fields
  tool_name: str
  
  # Optional fields
  agent_key: Optional[str] = None
  agent_role: Optional[str] = None

###Testing Improvements

1. Organization of Test Parameters
   • Current Implementation:

     def assert_event_count(mock_emit, expected_completed_tool_call: int = 0, ...):

   • Suggested Improvement:

     from typing import NamedTuple
     
     class ExpectedEventCounts(NamedTuple):
         completed_tool_call: int = 0
         ...
     def assert_event_count(mock_emit, expected_counts: ExpectedEventCounts, ...):

   • Reasoning: Using a NamedTuple can enhance parameter management and improve readability.

Documentation Suggestions

• Add comprehensive docstring comments detailing the new event classes and functionality. Include usage examples for clarity.

Security Recommendations

• Safeguard against logging sensitive information.
• Validate all tool arguments before execution to ensure security compliance.

Performance Considerations

• Although logging introduces some overhead, it is generally beneficial for observability. Debug-level logging should be implemented selectively in development to avoid excess log volume.

Historical Context and Related PRs

Given the implementation changes, it is vital to reference previous PRs that touched on event handling or tool execution to gather more insights on design decisions and potential pitfalls. This will also provide valuable lessons learned that could be applicable to this PR.

---

Overall, the changes in this PR are robust and notably improve the tool usage logging process within the codebase. Following the outlined recommendations for further refinement will bolster the code's maintainability and performance.

[lorenzejay]

lovely!

[lorenzejay]

nice