Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics collector - tool calls and requests #934

Merged

Conversation

EugeneLightsOn
Copy link
Collaborator

@EugeneLightsOn EugeneLightsOn commented Feb 5, 2025

Metrics collector

AI Description

This pull request introduces a new metrics system to the backend, enabling the collection and export of metrics data. The changes include:

  • Adding a new metrics directory to the .gitignore file.
  • Mounting the metrics directory in docker-compose.yml.
  • Modifying the CMD command in src/backend/Dockerfile to specify the port number as 8000.
  • Creating a new MetricsSettings class in src/backend/config/settings.py for managing metrics-related settings.
  • Importing RequestMetricsMiddleware and collector from backend.metrics in src/backend/main.py.
  • Adding a new backend.metrics module with several new files:
    • __init__.py: Imports RequestMetricsMiddleware, collector, MONITORED_PATHS, and track_tool_call_time from the backend.metrics package.
    • middleware.py: Defines the RequestMetricsMiddleware class for handling metrics collection and the MetricsCollector class for managing metrics data.
    • tool_call_decorator.py: Implements the track_tool_call_time decorator for tracking method execution time and logging it to the metrics collector.
  • Importing track_tool_call_time from backend.metrics in src/backend/tools/base.py and using it to decorate methods with the metrics collector and parameter checker.

@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2025

Codecov Report

Attention: Patch coverage is 48.52941% with 35 lines in your changes missing coverage. Please review.

Project coverage is 79.77%. Comparing base (83cd749) to head (e040b3b).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/backend/metrics/middleware.py 44.11% 19 Missing ⚠️
src/backend/metrics/tool_call_decorator.py 25.00% 12 Missing ⚠️
src/backend/main.py 50.00% 3 Missing ⚠️
src/backend/tools/base.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #934      +/-   ##
==========================================
- Coverage   79.95%   79.77%   -0.19%     
==========================================
  Files         292      295       +3     
  Lines       11898    11965      +67     
==========================================
+ Hits         9513     9545      +32     
- Misses       2385     2420      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@ezawadski ezawadski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better if we wrote to the file in real time? Instead of having to stop the app to get the metric data?

src/backend/Dockerfile Outdated Show resolved Hide resolved
@EugeneLightsOn EugeneLightsOn merged commit 2b30f73 into main Feb 6, 2025
8 checks passed
@EugeneLightsOn EugeneLightsOn deleted the eugene/tlk-2238-get-latency-metrics-on-caching-tools branch February 6, 2025 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants