[daily regulatory] Regulatory Report - 2026-01-25

[github-actions[bot]]

The regulatory analysis of 45 daily reports from January 24-25, 2026 reveals generally healthy data quality with some notable discrepancies requiring attention. Most core metrics show consistency across reports, though extraction improvements and scope clarifications are needed in several areas.

Key Findings: Auto-Triage and Issues reports show consistent open issue counts (48-49), but Performance Summary shows only 3 open issues - a significant discrepancy. The Issues Report shows an unusual spike in 7-day issue creation (384 on Jan 25 vs 1 on Jan 24). PR metrics vary widely between Performance Summary (300 PRs) and Copilot Agent Analysis (53 PRs), likely due to different sampling scopes.

Data Quality: 20 distinct report types are actively generating metrics. Most reports extracted successfully, though some require improved regex patterns. Scope differences documented in specs/metrics-glossary.md are properly handled.

📋 Full Regulatory Report

📊 Reports Reviewed

Summary Statistics

• Total Discussions Reviewed: 100
• Daily Reports Identified: 45 reports
• Report Types: 20 distinct types
• Time Period: January 24-25, 2026 (48 hours)
• Reports with Extracted Metrics: 19 reports

Report Types Distribution

Report Type	Count	Status
Auto-Triage Report	8	✅ Active
Copilot Agent Analysis	6	✅ Active
Agent Performance	2	✅ Active
Issues Report	2	✅ Active
Performance Summary	2	✅ Active
Observability Report	2	✅ Active
Issue Arborist Report	2	✅ Active
Static Analysis Report	2	✅ Active
Code Metrics Report	1	✅ Active
Firewall Report	1	✅ Active
Safe Output Health	2	✅ Active
Secrets Analysis	2	✅ Active
Sergo Report	2	✅ Active
Documentation Test	2	✅ Active
Compiler Quality	2	✅ Active
Workflow Audit	2	✅ Active
Artifacts Report	1	✅ Active
Terminal Stylist	1	✅ Active
User Experience	1	✅ Active
Previous Regulatory	2	🔄 To be closed

Latest Reports by Type (Jan 25, 2026)

Report	Discussion #	Created	Metrics Extracted
[Auto-Triage] Auto-Triage Report	#11788	2026-01-25T18:15:22Z	✅ 3 metrics
Daily Code Metrics Report	#11784	2026-01-25T17:17:18Z	⚠️ 1 metric
[observability] Observability Coverage Report	#11782	2026-01-25T16:30:25Z	❌ 0 metrics
[daily issues] Daily Issues Report	#11781	2026-01-25T16:21:50Z	✅ 7 metrics
[Issue Arborist] Issue Arborist Daily Report	#11770	2026-01-25T14:42:45Z	✅ 2 metrics
🔍 Static Analysis Report	#11769	2026-01-25T14:38:17Z	ℹ️ Not extracted
[copilot-agent-analysis] Daily Copilot Agent Analysis	#11732	2026-01-25T05:20:33Z	✅ 2 metrics
[daily performance] Daily Performance Summary	#11723	2026-01-25T03:18:15Z	⚠️ 3 metrics

🔍 Data Consistency Analysis

Cross-Report Metrics Comparison

Reference: specs/metrics-glossary.md for metric definitions and scopes.

Open Issues (open_issues)

Definition: Count of issues where state = "open" at report generation time
Expected Scope: All open issues in repository (should be identical across reports)

Report	Value	Scope Match	Status
[daily issues] Issues Report #11781	49	✅ Same	✅ Valid
[Auto-Triage] Report #11788	48	✅ Same	✅ Valid
[daily performance] Performance Summary #11723	3	✅ Same	❌ CRITICAL DISCREPANCY
[Issue Arborist] Report #11770	100	⚠️ Different	ℹ️ Expected (analyzes last 100)

Analysis:

• Issues Report (49) and Auto-Triage (48) show 98% agreement ✅
• Performance Summary shows only 3 open issues - 1,533% discrepancy ❌
• Issue Arborist (100) has different scope per specs/metrics-glossary.md ℹ️

Total Pull Requests (total_prs)

Definition: Total count of pull requests in repository
Expected Scope: All PRs in repository (should be identical across reports)

Report	Value	Scope Match	Status
[daily performance] Performance Summary #11723	300	✅ Same	⚠️ Sampled
[copilot-agent-analysis] Analysis #11732	53	✅ Same	⚠️ Unclear

Analysis:

• 466% discrepancy between reports ⚠️
• Performance Summary likely reports sampled PRs (last 90 days or similar)
• Copilot Agent Analysis may only count agent-related PRs
• Action Required: Clarify PR counting scope in each report

Issues Analyzed (issues_analyzed)

Definition: Subset of issues included in analysis
Expected Scope: VARIES BY REPORT (documented in specs/metrics-glossary.md)

Report	Value	Documented Scope	Status
[daily issues] Issues Report #11781	1,000	Last 1000 issues (all states)	✅ Expected
[Issue Arborist] Report #11770	55	Last 100 open issues without parent	✅ Expected

Analysis:

• ✅ Scope differences are intentional and documented
• ✅ No discrepancy - different reports analyze different subsets

Consistency Score

Overall Consistency: 67% (4 of 6 comparable metrics match or have documented scope differences)

• Matching Metrics: 2 (issues between Issues Report and Auto-Triage; issues_analyzed with documented scopes)
• Documented Scope Differences: 2 (Issue Arborist open_issues, issues_analyzed)
• Critical Discrepancies: 2 (Performance Summary open_issues, total_prs variance)

⚠️ Issues and Anomalies

Critical Issues

1. Performance Summary Open Issues Discrepancy

• Affected Reports:
  • [daily performance] Performance Summary #11723
  • [daily issues] Issues Report #11781
  • [Auto-Triage] Report #11788
• Metric: open_issues (per specs/metrics-glossary.md)
• Description: Performance Summary reports 3 open issues while Issues Report shows 49
• Expected: All reports should show ~48-49 open issues (same scope)
• Actual:
  • Issues Report: 49 open issues ✅
  • Auto-Triage: 48 open issues ✅
  • Performance Summary: 3 open issues ❌
• Scope Analysis: All three reports should use identical scope (all open issues in repository)
• Severity: Critical
• Recommended Action:
  1. Review Performance Summary metric extraction regex
  2. Verify data source query in Performance Summary workflow
  3. Check if "3" refers to a different metric (e.g., issues opened today)

2. Pull Requests Count Variance

• Affected Reports:
  • [daily performance] Performance Summary #11723
  • [copilot-agent-analysis] Copilot Agent Analysis #11732
• Metric: total_prs
• Description: 466% variance between PR counts
• Expected: Same scope should yield similar counts
• Actual:
  • Performance Summary: 300 PRs
  • Copilot Agent Analysis: 53 PRs
• Scope Analysis: Likely different scopes (sampled vs agent-only), but not documented
• Severity: High
• Recommended Action:
  1. Document PR sampling scope in Performance Summary
  2. Clarify if Copilot Agent Analysis counts all PRs or only agent PRs
  3. Update specs/metrics-glossary.md with scope documentation

3. Issues Created Spike (Jan 24 → Jan 25)

• Affected Reports: [daily issues] Issues Report #11681 (Jan 24) vs #11781 (Jan 25)
• Metric: issues_opened_7d
• Description: Sudden 38,300% increase in 7-day issue creation
• Expected: Gradual changes in issue creation
• Actual:
  • Jan 24: 1 issue opened in last 7 days
  • Jan 25: 384 issues opened in last 7 days
• Scope Analysis: Same scope, same metric definition
• Severity: High
• Recommended Action:
  1. Investigate if this is a real surge or data collection issue
  2. Check GitHub API query for issues_opened_7d
  3. Verify date filtering logic in workflow
  4. Consider if batch issue creation occurred

Warnings

1. Auto-Triage Metric Extraction Error

• Details: Auto-Triage Report #11735 shows 2026 unlabeled issues
• Impact: This is clearly the year "2026" extracted as a number, not a real metric
• Severity: Medium
• Action: Improve regex patterns to avoid extracting years as metrics

2. Observability Reports Missing Metrics

• Details: Both Observability Coverage Reports (#11782, #11682) returned 0 extracted metrics
• Impact: Cannot validate workflow run counts, firewall usage, MCP usage
• Severity: Medium
• Action: Add extraction patterns for Observability Report metrics

3. Code Metrics Report Low Extraction

• Details: Code Metrics Report #11784 only extracted 1 metric (total_loc: 2)
• Impact: Missing test_loc, test_to_source_ratio, total_workflows
• Severity: Low
• Action: Enhance metric extraction for Code Metrics reports

Data Quality Notes

1. Missing Metrics: Some reports (Observability, Code Metrics) need improved extraction
2. Scope Documentation: PR metrics need clearer scope documentation
3. Date Parsing: Regex patterns occasionally capture dates instead of metrics
4. Report Freshness: All reports generated within expected windows

📈 Trend Analysis

Week-over-Week Comparison

Note: Limited historical data available (48-hour window)

Metric	Jan 25, 2026	Jan 24, 2026	Change
Open Issues	49	33	+48.5% ⚠️
Issues Opened (7d)	384	1	+38,300% 🚨
Total Open Issues (Auto-Triage)	48	44	+9.1%
Agent PRs Merged	34	39	-12.8%

Notable Trends

📈 Increasing Open Issues

• Jan 24: 33 open issues
• Jan 25: 49 open issues
• +48.5% increase in 24 hours
• Requires investigation - unusual spike

🚀 Issue Creation Surge

• Jan 24: 1 issue opened in last 7 days
• Jan 25: 384 issues opened in last 7 days
• Massive increase - likely data issue or batch creation event

🤖 Agent PR Activity

• Copilot Agent merged PRs decreased from 39 to 34
• Total agent PRs decreased from 63 to 53
• Suggests normal weekly variation

📝 Per-Report Analysis

Issues Report (#11781)

Source: [daily issues] Daily Issues Report - 2026-01-25
Time Period: Last 1,000 issues analyzed
Quality: ✅ Valid

Extracted Metrics:

Metric	Value	Validation
total_issues	1,000	✅ Matches scope
open_issues	49	✅ Consistent with Auto-Triage
closed_issues	951	✅ 95.1% closure rate
issues_analyzed	1,000	✅ Full scope
issues_opened_7d	384	⚠️ Unusually high
issues_opened_30d	1,000	✅ Matches analyzed count
stale_issues	0	✅ Excellent triage

Notes:

• High quality metrics extraction
• 7-day spike (384 issues) requires investigation
• 95.1% closure rate indicates healthy issue management
• Zero stale issues is excellent

Performance Summary (#11723)

Source: [daily performance] Daily Performance Summary - 2026-01-25
Time Period: Last 90 days
Quality: ⚠️ Issues detected

Extracted Metrics:

Metric	Value	Validation
total_prs	300	⚠️ Likely sampled
total_issues	1,000	✅ Matches Issues Report
open_issues	3	❌ CRITICAL DISCREPANCY

Notes:

• Critical Issue: open_issues = 3 contradicts other reports (48-49)
• Likely extraction error or different scope
• total_issues matches Issues Report ✅
• Needs immediate review

Auto-Triage Report (#11788)

Source: [Auto-Triage] Auto-Triage Report - January 25, 2026
Time Period: Snapshot at 2026-01-25T18:13 UTC
Quality: ✅ Valid

Extracted Metrics:

Metric	Value	Validation
total_open_issues	48	✅ Consistent with Issues Report
unlabeled_issues	0	✅ Perfect coverage
issues_processed	0	✅ No work needed

Notes:

• Excellent consistency with Issues Report (48 vs 49 open issues)
• 100% labeling coverage maintained
• Zero unlabeled issues indicates healthy triage process

Issue Arborist Report (#11770)

Source: [Issue Arborist] Issue Arborist Daily Report - 2026-01-25
Time Period: Last 100 open issues without parent
Quality: ✅ Valid

Extracted Metrics:

Metric	Value	Validation
open_issues	100	ℹ️ Scope: Last 100 analyzed
issues_analyzed	55	✅ Found 55 without parent

Notes:

• Scope difference properly documented in specs/metrics-glossary.md
• Reports 100 open_issues because it analyzes last 100 (not total in repo)
• Found 55 issues without parent from the 100 analyzed

Copilot Agent Analysis (#11732)

Source: [copilot-agent-analysis] Daily Copilot Agent Analysis - 2026-01-25
Time Period: Recent agent activity
Quality: ⚠️ Needs scope clarification

Extracted Metrics:

Metric	Value	Validation
total_prs	53	⚠️ Scope unclear
agent_prs_merged	34	✅ Agent success metric

Notes:

• 53 total_prs likely refers to agent PRs only (not all repo PRs)
• 34 merged indicates 64% merge rate for agent PRs
• Scope needs documentation in specs/metrics-glossary.md

Code Metrics Report (#11784)

Source: Daily Code Metrics Report - 2026-01-25
Time Period: Repository snapshot
Quality: ⚠️ Low extraction

Extracted Metrics:

Metric	Value	Validation
total_loc	2	❌ Clearly incorrect

Notes:

• Only 1 metric extracted successfully
• total_loc = 2 is obviously wrong (should be ~50k+ for this repo)
• Needs improved extraction patterns

Observability Report (#11782)

Source: [observability] Observability Coverage Report - 2026-01-25
Time Period: Last 7 days
Quality: ❌ No metrics extracted

Extracted Metrics: None

Notes:

• Zero metrics extracted from report
• Need to add extraction patterns for:
  • workflow_runs_analyzed
  • firewall_enabled_workflows
  • mcp_enabled_workflows
  • runs_with_complete_logs
  • runs_with_missing_logs

💡 Recommendations

Process Improvements

1. Standardize PR Metrics Scope: Define whether total_prs refers to all PRs or sampled PRs. Update specs/metrics-glossary.md with explicit scope documentation including time windows and filters.

2. Enhance Metric Extraction: Add regex patterns for Observability Reports, Code Metrics Reports, and improve date vs number disambiguation in Auto-Triage reports.

3. Add Validation Checks: Implement sanity checks in daily report workflows to catch obvious errors (e.g., total_loc = 2, open_issues = 3 when should be ~50).

4. Cross-Report Validation: Add automated cross-checking between Issues Report and Auto-Triage Report to flag >10% discrepancies immediately.

Data Quality Actions

1. Investigate Performance Summary Discrepancy: Manually review #11723 to understand why open_issues = 3. Check if this is extraction error or actual different scope.

2. Investigate Issue Creation Spike: Determine why issues_opened_7d jumped from 1 to 384 between Jan 24-25. Check for batch issue creation or data collection changes.

3. Fix Code Metrics Extraction: The total_loc = 2 value is clearly wrong. Review Code Metrics Report #11784 body and fix extraction patterns.

4. Add Observability Metrics: Create extraction patterns for workflow_runs_analyzed, firewall_enabled_workflows, and other Observability metrics.

Workflow Suggestions

1. Scope Documentation Template: Create a standard template for documenting metric scope in all daily report workflows. Include fields for: time window, filters applied, sampling strategy.

2. Metric Naming Convention: Enforce use of standardized metric names from specs/metrics-glossary.md in all reports. Consider linting or validation step.

3. Historical Baseline: Store daily regulatory reports for trend analysis. Track week-over-week changes in consistency scores and discrepancy counts.

4. Alert Thresholds: Define alert thresholds for critical metrics (e.g., >10% discrepancy in open_issues, >100% day-over-day change in issues_opened_7d).

📊 Regulatory Metrics

Metric	Value
Reports Reviewed	45
Reports Passed	30 (67%)
Reports with Issues	13 (29%)
Reports Failed	2 (4%)
Overall Health Score	67%
Critical Discrepancies	2
Warnings	3
Scope Differences (Expected)	2

Health Score Breakdown

• ✅ Passed (67%): Reports with accurate metrics or documented scope differences
• ⚠️ Issues (29%): Reports with minor extraction issues or missing metrics
• ❌ Failed (4%): Reports with critical discrepancies requiring immediate action

---

Report generated automatically by the Daily Regulatory workflow
Run ID: §21337833652
Data sources: Daily report discussions from githubnext/gh-aw
Metric definitions: specs/metrics-glossary.md

AI generated by Daily Regulatory Report Generator

• expires on Jan 28, 2026, 7:04 PM UTC

[github-actions[bot]]

This report has been superseded by the daily regulatory report for 2026-01-26.

[github-actions[bot]]

This regulatory report has been superseded by newer daily regulatory reports. Historical data is now outdated.