Skip to content

Commit 2c4af80

Browse files
MervinPraisonMervinPraison
committed
Enhance telemetry support and update dependencies
- Added 'posthog>=4.0.0' to dependencies and optional 'telemetry' section in pyproject.toml. - Updated blog_agent to use llm_config for improved clarity. - Disabled memory in PraisonAIAgents instantiation for better control. - Introduced telemetry support with lazy loading and fallback functions in __init__.py. This update improves the integration of telemetry features while maintaining existing functionality.
1 parent dc45059 commit 2c4af80

29 files changed

+3195
-5
lines changed

‎src/praisonai-agents/TELEMETRY_SUMMARY.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Telemetry Implementation Summary
2+
3+
## What Was Fixed
4+
5+
1. **PostHog initialization error** - Removed invalid `events_to_ignore` parameter
6+
2. **Missing imports** - Added `MinimalTelemetry` and `TelemetryCollector` imports to telemetry `__init__.py`
7+
3. **Wrong method instrumentation** - Changed from `agent.execute()` to `agent.chat()`
8+
4. **Task tracking** - Added instrumentation for `workflow.execute_task()`
9+
5. **Automatic setup** - Added `auto_instrument_all()` in main `__init__.py`
10+
6. **Automatic flush** - Added `atexit` handler to send data on program exit
11+
12+
## Current Status
13+
14+
✅ **Telemetry is now working automatically!**
15+
16+
- Enabled by default (opt-out via environment variables)
17+
- Tracks agent executions and task completions
18+
- Sends anonymous data to PostHog on program exit
19+
- No manual setup required
20+
21+
## Metrics Example
22+
```
23+
Telemetry metrics collected:
24+
- Agent executions: 4
25+
- Task completions: 2
26+
- Errors: 0
27+
- Session ID: 33873e62396d8b4c
28+
```
29+
30+
## To Disable
31+
Set any of these environment variables:
32+
- `PRAISONAI_TELEMETRY_DISABLED=true`
33+
- `DO_NOT_TRACK=true`

‎src/praisonai-agents/debug_auto_instrument.py‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Debug auto-instrumentation.
4+
"""
5+
6+
print("1. Import telemetry module...")
7+
import praisonaiagents.telemetry
8+
print(" Telemetry module imported")
9+
10+
print("\n2. Check if auto_instrument_all was called...")
11+
print(f" _initialized: {praisonaiagents.telemetry._initialized}")
12+
13+
print("\n3. Import Agent and PraisonAIAgents...")
14+
from praisonaiagents import Agent, PraisonAIAgents
15+
print(" Classes imported")
16+
17+
print("\n4. Check if classes are instrumented...")
18+
agent = Agent(name="Test", role="Test", goal="Test", instructions="Test")
19+
print(f" Agent.__init__ name: {Agent.__init__.__name__}")
20+
print(f" agent.execute exists: {hasattr(agent, 'execute')}")
21+
22+
print("\n5. Manually call auto_instrument_all...")
23+
from praisonaiagents.telemetry.integration import auto_instrument_all
24+
auto_instrument_all()
25+
print(" auto_instrument_all() called")
26+
27+
print("\n6. Create new agent after instrumentation...")
28+
agent2 = Agent(name="Test2", role="Test2", goal="Test2", instructions="Test2")
29+
print(f" Agent.__init__ name after: {Agent.__init__.__name__}")
30+
print(f" agent2.execute exists: {hasattr(agent2, 'execute')}")
31+
32+
print("\n7. Check if execute is wrapped...")
33+
if hasattr(agent2, 'execute'):
34+
print(f" agent2.execute name: {agent2.execute.__name__}")
35+
print(f" agent2.execute wrapped: {hasattr(agent2.execute, '__wrapped__')}")
36+
37+
print("\n8. Import telemetry and check if it's working...")
38+
from praisonaiagents.telemetry import get_telemetry
39+
telemetry = get_telemetry()
40+
print(f" Telemetry enabled: {telemetry.enabled}")
41+
print(f" PostHog available: {telemetry._posthog is not None}")
42+
43+
# The key insight: auto_instrument_all needs to be called AFTER
44+
# the Agent and PraisonAIAgents classes are imported!

‎src/praisonai-agents/debug_telemetry.py‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Debug telemetry instrumentation to see what's happening.
4+
"""
5+
6+
import os
7+
# Make sure telemetry is enabled
8+
if 'PRAISONAI_TELEMETRY_DISABLED' in os.environ:
9+
del os.environ['PRAISONAI_TELEMETRY_DISABLED']
10+
11+
print("1. Importing modules...")
12+
from praisonaiagents import Agent, Task, PraisonAIAgents
13+
from praisonaiagents.telemetry import get_telemetry
14+
15+
print("\n2. Checking telemetry status...")
16+
telemetry = get_telemetry()
17+
print(f"Telemetry enabled: {telemetry.enabled}")
18+
print(f"PostHog available: {telemetry._posthog is not None}")
19+
20+
print("\n3. Creating agent...")
21+
agent = Agent(
22+
name="TestAgent",
23+
role="Test Role",
24+
goal="Test Goal",
25+
instructions="Test instructions"
26+
)
27+
28+
# Check if agent.execute is instrumented
29+
print(f"\n4. Checking agent instrumentation...")
30+
print(f"Agent has execute method: {hasattr(agent, 'execute')}")
31+
if hasattr(agent, 'execute'):
32+
print(f"Execute method type: {type(agent.execute)}")
33+
print(f"Execute method name: {agent.execute.__name__ if hasattr(agent.execute, '__name__') else 'No name'}")
34+
print(f"Is wrapped: {'instrumented' in str(agent.execute.__name__) if hasattr(agent.execute, '__name__') else 'Unknown'}")
35+
36+
print("\n5. Creating task...")
37+
task = Task(
38+
description="Test task",
39+
expected_output="Test output",
40+
agent=agent
41+
)
42+
43+
print("\n6. Creating workflow...")
44+
workflow = PraisonAIAgents(
45+
agents=[agent],
46+
tasks=[task],
47+
process="sequential"
48+
)
49+
50+
# Check if workflow.start is instrumented
51+
print(f"\n7. Checking workflow instrumentation...")
52+
print(f"Workflow has start method: {hasattr(workflow, 'start')}")
53+
if hasattr(workflow, 'start'):
54+
print(f"Start method type: {type(workflow.start)}")
55+
print(f"Start method name: {workflow.start.__name__ if hasattr(workflow.start, '__name__') else 'No name'}")
56+
print(f"Is wrapped: {'instrumented' in str(workflow.start.__name__) if hasattr(workflow.start, '__name__') else 'Unknown'}")
57+
58+
print("\n8. Running workflow...")
59+
result = workflow.start()
60+
61+
print("\n9. Checking metrics...")
62+
metrics = telemetry.get_metrics()
63+
print(f"Metrics: {metrics}")
64+
65+
print("\n10. Manually tracking to verify telemetry works...")
66+
telemetry.track_agent_execution("ManualTest", success=True)
67+
telemetry.track_task_completion("ManualTask", success=True)
68+
manual_metrics = telemetry.get_metrics()
69+
print(f"After manual tracking: {manual_metrics['metrics']}")

‎src/praisonai-agents/debug_telemetry_double.py‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Debug double-counting in telemetry.
4+
"""
5+
6+
from praisonaiagents import Agent, Task, PraisonAIAgents
7+
from praisonaiagents.telemetry import get_telemetry
8+
9+
# Get telemetry instance
10+
telemetry = get_telemetry()
11+
12+
# Clear any existing metrics by flushing
13+
telemetry.flush()
14+
15+
print("Starting fresh telemetry tracking...\n")
16+
17+
# Create ONE agent
18+
print("Creating 1 agent...")
19+
agent = Agent(
20+
name="SingleAgent",
21+
role="Test Role",
22+
goal="Test Goal",
23+
instructions="Test instructions"
24+
)
25+
26+
# Create ONE task
27+
print("Creating 1 task...")
28+
task = Task(
29+
description="Single test task",
30+
expected_output="Test output",
31+
agent=agent
32+
)
33+
34+
# Create workflow with ONE agent and ONE task
35+
print("Creating workflow with 1 agent and 1 task...")
36+
workflow = PraisonAIAgents(
37+
agents=[agent],
38+
tasks=[task],
39+
process="sequential"
40+
)
41+
42+
# Check metrics before running
43+
metrics_before = telemetry.get_metrics()
44+
print(f"\nMetrics BEFORE running workflow:")
45+
print(f" Agent executions: {metrics_before['metrics']['agent_executions']}")
46+
print(f" Task completions: {metrics_before['metrics']['task_completions']}")
47+
48+
# Run the workflow
49+
print("\nRunning workflow...")
50+
result = workflow.start()
51+
52+
# Check metrics after running
53+
metrics_after = telemetry.get_metrics()
54+
print(f"\nMetrics AFTER running workflow:")
55+
print(f" Agent executions: {metrics_after['metrics']['agent_executions']} (expected: 1)")
56+
print(f" Task completions: {metrics_after['metrics']['task_completions']} (expected: 1)")
57+
58+
if metrics_after['metrics']['agent_executions'] > 1:
59+
print("\n❌ ISSUE: Agent executions are being double-counted!")
60+
print(" Possible causes:")
61+
print(" - Agent method is being called multiple times")
62+
print(" - Instrumentation is being applied twice")
63+
print(" - Multiple tracking calls for same execution")
64+
65+
if metrics_after['metrics']['task_completions'] > 1:
66+
print("\n❌ ISSUE: Task completions are being double-counted!")
67+
print(" Possible causes:")
68+
print(" - Task completion is tracked in multiple places")
69+
print(" - Instrumentation is being applied twice")

‎src/praisonai-agents/praisonaiagents/__init__.py‎

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,50 @@
3030
async_display_callbacks,
3131
)
3232

33+
# Telemetry support (lazy loaded)
34+
try:
35+
from .telemetry import (
36+
get_telemetry,
37+
enable_telemetry,
38+
disable_telemetry,
39+
MinimalTelemetry,
40+
TelemetryCollector
41+
)
42+
_telemetry_available = True
43+
except ImportError:
44+
# Telemetry not available - provide stub functions
45+
_telemetry_available = False
46+
def get_telemetry():
47+
return None
48+
49+
def enable_telemetry(*args, **kwargs):
50+
import logging
51+
logging.warning(
52+
"Telemetry not available. Install with: pip install praisonaiagents[telemetry]"
53+
)
54+
return None
55+
56+
def disable_telemetry():
57+
pass
58+
59+
MinimalTelemetry = None
60+
TelemetryCollector = None
61+
3362
# Add Agents as an alias for PraisonAIAgents
3463
Agents = PraisonAIAgents
3564

65+
# Apply telemetry auto-instrumentation after all imports
66+
if _telemetry_available:
67+
try:
68+
# Only instrument if telemetry is enabled
69+
_telemetry = get_telemetry()
70+
if _telemetry and _telemetry.enabled:
71+
from .telemetry.integration import auto_instrument_all
72+
auto_instrument_all(_telemetry)
73+
except Exception:
74+
# Silently fail if there are any issues
75+
pass
76+
3677
__all__ = [
3778
'Agent',
3879
'ImageAgent',
@@ -60,5 +101,10 @@
60101
'Chunking',
61102
'MCP',
62103
'GuardrailResult',
63-
'LLMGuardrail'
104+
'LLMGuardrail',
105+
'get_telemetry',
106+
'enable_telemetry',
107+
'disable_telemetry',
108+
'MinimalTelemetry',
109+
'TelemetryCollector'
64110
]

‎src/praisonai-agents/praisonaiagents/telemetry/README.md‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# PraisonAI Agents Telemetry
2+
3+
This module provides minimal, privacy-focused telemetry for PraisonAI Agents.
4+
5+
## Privacy Guarantees
6+
7+
- **No personal data is collected** - No prompts, responses, or user content
8+
- **Anonymous metrics only** - Usage counts and feature adoption
9+
- **Opt-out by default** - Respects standard privacy preferences
10+
- **Transparent collection** - See exactly what's tracked below
11+
12+
## What We Collect
13+
14+
We collect only anonymous usage metrics:
15+
- Number of agent executions
16+
- Number of task completions
17+
- Tool usage (names only, no arguments)
18+
- Error types (no error messages)
19+
- Framework version and OS type
20+
- Anonymous session ID (regenerated each run)
21+
22+
## Disabling Telemetry
23+
24+
Telemetry can be disabled in three ways:
25+
26+
### 1. Environment Variables (Recommended)
27+
28+
Set any of these environment variables:
29+
```bash
30+
export PRAISONAI_TELEMETRY_DISABLED=true
31+
export PRAISONAI_DISABLE_TELEMETRY=true
32+
export DO_NOT_TRACK=true # Universal standard
33+
```
34+
35+
### 2. Programmatically
36+
37+
```python
38+
from praisonaiagents.telemetry import disable_telemetry
39+
disable_telemetry()
40+
```
41+
42+
### 3. At Runtime
43+
44+
```python
45+
from praisonaiagents.telemetry import get_telemetry
46+
telemetry = get_telemetry()
47+
telemetry.enabled = False
48+
```
49+
50+
## Usage
51+
52+
The telemetry module integrates automatically with PraisonAI Agents:
53+
54+
```python
55+
from praisonaiagents import Agent, Task, PraisonAIAgents
56+
57+
# Telemetry is automatically enabled (unless disabled by environment)
58+
agent = Agent(name="MyAgent", role="Assistant")
59+
task = Task(description="Help user", agent=agent)
60+
61+
workflow = PraisonAIAgents(agents=[agent], tasks=[task])
62+
result = workflow.start()
63+
64+
# Check telemetry metrics
65+
from praisonaiagents.telemetry import get_telemetry
66+
telemetry = get_telemetry()
67+
print(telemetry.get_metrics())
68+
```
69+
70+
## Implementation Details
71+
72+
The telemetry implementation is minimal and lightweight:
73+
- No external dependencies required
74+
- No network calls in current implementation
75+
- Metrics stored in memory only
76+
- Future versions may send to PostHog or similar privacy-focused services
77+
78+
## Backward Compatibility
79+
80+
The module maintains compatibility with the previous telemetry interface:
81+
82+
```python
83+
from praisonaiagents.telemetry import TelemetryCollector
84+
85+
collector = TelemetryCollector()
86+
collector.start()
87+
88+
with collector.trace_agent_execution("MyAgent"):
89+
# Agent execution code
90+
pass
91+
92+
collector.stop()
93+
```
94+
95+
## Contributing
96+
97+
When contributing to telemetry:
98+
1. Never collect personal data or user content
99+
2. Always make new metrics opt-out
100+
3. Document what's collected
101+
4. Keep the implementation minimal
102+
103+
## Future Plans
104+
105+
- Integration with PostHog for anonymous analytics
106+
- Aggregate usage statistics dashboard
107+
- Opt-in detailed performance metrics
108+
- Self-hosted telemetry endpoint option

0 commit comments

Comments
 (0)