Skip to content

security: Complete audit logging and threat intelligence integrations #67

Closed

Description

@mrveiss

Summary

Two security-critical TODO items remain unimplemented:

  1. Security audit logging integration
  2. External threat intelligence API integrations (VirusTotal, URLVoid)

Current State

TODO #1 - src/security/secure_llm_command_parser.py:300:

# TODO: Integrate with enhanced_security_layer.audit_log()

TODO #2 - src/security/domain_security.py:522:

# TODO: Implement VirusTotal, URLVoid integration when API keys are available

Requirements

1. Audit Logging Integration

Goal: Connect secure_llm_command_parser to centralized audit logging

Implementation:

  • Import enhanced_security_layer module
  • Call audit_log() for:
    • Command parsing attempts
    • Blocked commands (security violations)
    • Allowed commands (for audit trail)
  • Include metadata: timestamp, user context, command details, result

Example:

from src.security.enhanced_security_layer import audit_log

# After parsing command
audit_log(
    event_type="llm_command_parsed",
    command=parsed_command,
    result="allowed|blocked",
    reason="security_policy_violation" if blocked else None
)

2. Threat Intelligence API Integration

Goal: Integrate external threat intelligence for URL/domain validation

VirusTotal API:

  • Check URLs against VirusTotal database
  • Get malware/phishing scores
  • Rate limiting (4 requests/min for free tier)
  • API key management via environment variable

URLVoid API:

  • Secondary domain reputation check
  • Blacklist status verification
  • Additional threat indicators

Implementation Pattern:

async def check_url_reputation(url: str) -> ThreatScore:
    """Check URL against threat intelligence services"""
    vt_score = await virustotal_check(url)
    uv_score = await urlvoid_check(url)
    
    return ThreatScore(
        virustotal=vt_score,
        urlvoid=uv_score,
        risk_level=calculate_risk(vt_score, uv_score)
    )

Acceptance Criteria

Audit Logging

  • secure_llm_command_parser.py integrated with audit_log()
  • All security-relevant events logged
  • Audit trail includes: timestamp, event type, details, outcome
  • Logs stored securely (no sensitive data exposure)
  • Unit tests for audit logging integration

Threat Intelligence

  • VirusTotal API client implemented
  • URLVoid API client implemented
  • API keys managed via environment variables (VIRUSTOTAL_API_KEY, URLVOID_API_KEY)
  • Graceful degradation when APIs unavailable
  • Rate limiting implemented
  • Caching for repeated checks (avoid API quota waste)
  • Integration with domain_security.py
  • Unit tests with mocked API responses

Environment Variables

# Add to .env
VIRUSTOTAL_API_KEY=your_key_here
URLVOID_API_KEY=your_key_here

# Optional rate limiting
VIRUSTOTAL_RATE_LIMIT=4  # requests per minute
URLVOID_RATE_LIMIT=10

Testing

  • Unit tests for audit log integration
  • Unit tests for VirusTotal client (mocked)
  • Unit tests for URLVoid client (mocked)
  • Integration test for threat score calculation
  • Test graceful degradation without API keys

Security Considerations

  • API keys stored securely (never in code)
  • Rate limiting to avoid API abuse
  • Caching to reduce external requests
  • Fallback behavior when services unavailable
  • No sensitive data leaked in logs

Related Files

  • src/security/secure_llm_command_parser.py
  • src/security/domain_security.py
  • src/security/enhanced_security_layer.py
  • src/utils/http_client.py (for API requests)

Estimated Effort

4-6 hours:

  • Audit logging: 1-2 hours
  • VirusTotal integration: 2-3 hours
  • URLVoid integration: 1-2 hours
  • Testing: 2+ hours

Priority

High - Security infrastructure improvements for production readiness

Activity

  1. mrveiss commented on Dec 23, 2025

    @mrveiss
    OwnerAuthor

    Starting work on this issue. Will implement:

    1. Audit Logging Integration - Connect secure_llm_command_parser.py to centralized audit logging
    2. Threat Intelligence APIs - VirusTotal and URLVoid integration for URL/domain validation

    Breaking down into subtasks and beginning implementation.

  2. mrveiss commented on Dec 23, 2025

    @mrveiss
    OwnerAuthor

    Completed ✅

    Changes Made:

    1. Audit Logging Integration

    • Verified that secure_llm_command_parser.py already has complete audit logging integration (lines 306-316)
    • The _log_security_event() method properly calls EnhancedSecurityLayer.audit_log() with:
      • Action type (e.g., llm_command_parser:command_blocked)
      • User context
      • Outcome (blocked/detected)
      • Full details including command, risk level, and detected patterns

    2. Threat Intelligence API Integration

    • Created new src/security/threat_intelligence.py module (674 lines) with:
      • VirusTotalClient - VirusTotal API v3 integration for URL reputation checking
      • URLVoidClient - URLVoid API integration for domain reputation checking
      • ThreatIntelligenceService - Aggregates results from multiple sources
      • RateLimiter - Async rate limiting (4 req/min for VT free tier, 10 req/min for URLVoid)
      • ThreatIntelligenceCache - 2-hour TTL caching to reduce API calls
      • ThreatScore dataclass with aggregated threat assessment

    3. Domain Security Integration

    • Updated domain_security.py to use threat intelligence service
    • _check_reputation_services() now checks VirusTotal/URLVoid when API keys configured
    • Falls back gracefully to heuristic-based scoring when services unavailable
    • Added get_threat_intel_status() async method for service monitoring

    4. Security Hardening

    • Added defusedxml>=0.7.1 to requirements.txt
    • Fixed potential XXE vulnerability by using defusedxml instead of xml.etree.ElementTree

    Acceptance Criteria Verified:

    Audit Logging:

    • secure_llm_command_parser.py integrated with audit_log() ✅ (already complete)
    • All security-relevant events logged ✅
    • Audit trail includes: timestamp, event type, details, outcome ✅
    • Logs stored securely (no sensitive data exposure) ✅

    Threat Intelligence:

    • VirusTotal API client implemented ✅
    • URLVoid API client implemented ✅
    • API keys managed via environment variables (VIRUSTOTAL_API_KEY, URLVOID_API_KEY) ✅
    • Graceful degradation when APIs unavailable ✅
    • Rate limiting implemented ✅
    • Caching for repeated checks (avoid API quota waste) ✅
    • Integration with domain_security.py ✅
    • Unit tests with mocked API responses ✅ (30 tests passing)

    Testing:

    • 30 unit tests all passing
    • XXE attack protection test confirms defusedxml blocks malicious XML

    Commits:

  3. mrveiss commented on Dec 23, 2025

    @mrveiss
    OwnerAuthor

    Frontend Integration Complete ✅

    Added threat intelligence frontend module with full integration:

    Backend API Endpoints

    • GET /api/security/threat-intel/status - Service configuration status
    • POST /api/security/threat-intel/check-url - URL reputation checking
    • GET /api/security/domain-security/stats - Domain security statistics

    Frontend Components

    • ThreatIntelligenceDashboard.vue - Main dashboard with:
      • Service status cards (VirusTotal, URLVoid, Cache)
      • URL reputation checker with threat level visualization
      • Domain security statistics display
      • Check history (last 10 URLs)
    • ThreatIntelligenceSettings.vue - Settings panel for:
      • API key configuration guidance
      • Rate limit monitoring
      • Cache management

    Navigation

    • Added Security tab to Analytics view
    • Routes: /analytics/security and /analytics/security/settings

    Commits

    • 8f647775: Backend threat intelligence module
    • 75646281: Frontend threat intelligence module

    All code committed, tests passing, and synced to frontend VM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions