Skip to content

fix: add LRU cache with size limits to prevent memory exhaustion (#39)#40

Merged
willibrandon merged 3 commits intomainfrom
fix/template-cache-lru-limit
Aug 13, 2025
Merged

fix: add LRU cache with size limits to prevent memory exhaustion (#39)#40
willibrandon merged 3 commits intomainfrom
fix/template-cache-lru-limit

Conversation

@willibrandon
Copy link
Owner

@willibrandon willibrandon commented Aug 13, 2025

Description

Replaces the unbounded template cache with a thread-safe, sharded LRU cache implementation to prevent memory exhaustion from dynamic template generation. The cache enforces strict size limits while maintaining high performance through lock sharding and atomic statistics.

The previous implementation used an unbounded map that could grow indefinitely when applications generated templates dynamically (e.g., fmt.Sprintf("User %d: {Action}", id)), creating a potential DoS vulnerability.

Type of change

  • Bug fix
  • New feature
  • Performance improvement
  • Documentation update

Checklist

  • Tests pass (go test ./...)
  • Linter passes (golangci-lint run)
  • Benchmarks checked (if performance-related)
  • Documentation updated (if needed)
  • Zero-allocation promise maintained (if applicable)

Additional notes

Performance metrics:

  • Get operation: 57ns/op, 13B/op, 1 alloc
  • Put operation: 82ns/op, 97B/op, 5 allocs
  • Cache hits: 67ns/op, 0B/op, 0 allocs ✓

Security validation:

Test confirms 1,000,000 unique templates are safely bounded to configured max size (100 entries), with 999,900 evictions and minimal memory growth (<100KB).

Key implementation details:

  • 64 shards (dynamic, up to) with power-of-2 masking for fast modulo
  • FNV-1a hash with bit mixing for optimal distribution
  • Exact capacity distribution across shards (no over-allocation)
  • Atomic statistics for lock-free monitoring
  • One-time global configuration to prevent runtime reconfiguration issues

Fixes #39

Replace unbounded template cache with sharded LRU implementation:
- Configurable max size (default: 10,000 templates)
- O(1) get/put operations with proper eviction
- 64 shards with dynamic sizing based on capacity
- Atomic stats tracking (hits/misses/evictions/expirations)
- Optional TTL support with lazy cleanup
- Thread-safe global cache with one-time configuration

Security: Fixes potential DoS via dynamic template generation that could cause unbounded memory growth. Cache now strictly enforces size limits.

Performance: 57ns/op for Get, 82ns/op for Put, zero allocations on hits. Benchmarks show 100% hit rate for repeated templates.

Testing: Added tests including specific validation of the memory exhaustion scenario (1M unique templates → 100 cached entries).
@willibrandon willibrandon self-assigned this Aug 13, 2025
@willibrandon willibrandon added enhancement New feature or request security Security improvements and vulnerability fixes labels Aug 13, 2025
- Document template cache configuration in README.md
- Add cache implementation details to CLAUDE.md (v0.8.1+)
- Create examples/template-cache demonstrating usage and protection
- Explain security fix for memory exhaustion vulnerability
@willibrandon willibrandon requested a review from Copilot August 13, 2025 04:08
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR replaces the unbounded template cache with a thread-safe, sharded LRU cache implementation to prevent memory exhaustion from dynamic template generation. The new cache enforces strict size limits while maintaining high performance through lock sharding and atomic statistics, fixing a potential DoS vulnerability where applications generating templates dynamically could cause unbounded memory growth.

Key Changes

  • Implemented sharded LRU cache with configurable size limits and optional TTL
  • Replaced simple map-based cache with bounded, thread-safe implementation
  • Added comprehensive test coverage including security tests and benchmarks

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
internal/parser/lru_cache.go Core LRU cache implementation with sharding, TTL support, and thread-safe operations
internal/parser/lru_cache_test.go Comprehensive test suite covering basic operations, concurrency, eviction, and benchmarks
internal/parser/cache_security_test.go Security-focused tests validating protection against memory exhaustion attacks
internal/parser/cache.go Updated cache interface to use new LRU implementation instead of unbounded map
examples/template-cache/main.go Example demonstrating cache configuration and monitoring capabilities
README.md Documentation updates explaining template cache configuration and monitoring
CLAUDE.md Development notes documenting the new template cache feature

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

- Check globalCacheConfigured flag in initGlobalCache to prevent race
- Replace magic number with calculated constant in memory test
- Optimize time conversions using direct nanosecond arithmetic (2 places)
@willibrandon willibrandon merged commit 1cdcc7d into main Aug 13, 2025
56 of 124 checks passed
@willibrandon willibrandon deleted the fix/template-cache-lru-limit branch August 13, 2025 04:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request security Security improvements and vulnerability fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Template cache needs protection against unbounded growth

1 participant