Feedback-Driven Learning System

A lightweight, feedback-driven learning system for AI-assisted development. Close the loop between human feedback and AI code generation quality through systematic pattern recognition and guideline evolution.

🎯 Purpose

Transform human feedback into actionable patterns that improve AI assistance over time. Instead of repeating the same mistakes, your AI assistant learns from corrections and applies insights to future generations.

🔄 How It Works

Human Feedback → Pattern Extraction → Guideline Updates → Better AI Generations
     ↑                                                      ↓
     ←← Effect Tracking ← Apply in Practice ← Confidence Updates ← Success Tracking ←←

🚀 Quick Start

1. Initialize the System

# Clone and setup
git clone <your-repo>
cd learning-system

# Initialize local database
sqlite3 metrics.sqlite < schema.sql

# Verify setup
sqlite3 metrics.sqlite "SELECT 'Setup complete!' as status;"

2. Record Your First Feedback

# Example: AI generated code that needed correction
sqlite3 metrics.sqlite "
INSERT INTO feedback_sessions (
  feedback_type, human_rating, scenario, file_path, 
  feedback_text, correction_provided, correction_code
) VALUES (
  'failure', 4, 'controller-logic', 'app/Controllers/UserController.php',
  'Controller contains business logic that should be in service layer',
  1, 'Move calculation to UserService; inject into controller'
);"

3. Run Daily Analysis

# Generate insights
sqlite3 metrics.sqlite < queries.sql

# View learning curve
sqlite3 metrics.sqlite "SELECT * FROM learning_curve;"

# Check pattern effectiveness
sqlite3 metrics.sqlite "SELECT * FROM pattern_effectiveness;"

📊 Core Features

Feedback Classification

• Success: AI generated exactly what you needed
• Failure: Significant corrections required
• Partial: Mostly correct with minor adjustments
• Suggestion: Ideas for improvement

Priority Ranking

• P0 (Critical): ≥2 failures in 7 days, avg rating ≤5
• P1 (High): ≥3 issues in 14 days, avg rating ≤6 OR correction rate ≥50%
• P2 (Medium): ≥2 issues in 30 days, avg rating ≤7
• P3 (Low): Everything else

Pattern Categories

• Architectural: Structure and design decisions
• Syntax: Language-specific formatting and conventions
• Business Logic: Domain-specific rules and calculations
• UI/UX: User interface and experience patterns
• Needs Analysis: Requirements understanding gaps

📋 Database Schema

Core Tables

feedback_sessions: Records all human feedback

• feedback_type: success|failure|partial|suggestion
• human_rating: 1-10 satisfaction score
• scenario: Context category (e.g., 'api-design', 'data-validation')
• correction_code: How you fixed the AI's output

learned_patterns: Extracted patterns from feedback

• trigger_conditions: When this pattern applies (JSON)
• symptom_description: What went wrong
• solution_template: How to fix it
• confidence_score: 0.0-1.0 reliability
• success_count/application_count: Track effectiveness

Helpful Views

learning_curve: Daily progress metrics

SELECT * FROM learning_curve WHERE date >= date('now', '-30 days');

pattern_effectiveness: Success rates by category

SELECT * FROM pattern_effectiveness ORDER BY success_rate DESC;

🛠 Usage Examples

Web Development

# Frontend component issue
sqlite3 metrics.sqlite "
INSERT INTO feedback_sessions (feedback_type, human_rating, scenario, feedback_text, correction_code)
VALUES ('partial', 6, 'react-component', 'Missing error boundary wrapper', 'Wrap component with <ErrorBoundary>');"

API Development

# Backend validation issue
sqlite3 metrics.sqlite "
INSERT INTO feedback_sessions (feedback_type, human_rating, scenario, feedback_text, correction_code)
VALUES ('failure', 3, 'input-validation', 'No request validation in controller', 'Add FormRequest class with validation rules');"

Data Processing

# Algorithm optimization
sqlite3 metrics.sqlite "
INSERT INTO feedback_sessions (feedback_type, human_rating, scenario, feedback_text, correction_code)
VALUES ('suggestion', 8, 'performance', 'Could use more efficient sorting algorithm', 'Replace bubble sort with quicksort for large datasets');"

📈 Daily Workflow

1. Collect Feedback

As you work with AI assistance, rate outputs 1-10 and note corrections:

# Quick feedback entry
./scripts/add_feedback.sh "failure" 4 "database-query" "Missing JOIN clause" "Add LEFT JOIN users table"

2. Process High-Priority Items

# Find critical issues needing attention
sqlite3 metrics.sqlite < queries.sql | jq '.[] | select(.rank == "P0")'

3. Update Patterns

Review and approve AI-suggested patterns:

# Check unprocessed feedback with corrections
sqlite3 metrics.sqlite "
SELECT scenario, feedback_text, correction_code 
FROM feedback_sessions 
WHERE processed = 0 AND correction_provided = 1 
ORDER BY human_rating ASC
LIMIT 5;"

4. Export Insights

# Generate daily report
mkdir -p reports/daily
sqlite3 metrics.sqlite < queries.sql > "reports/daily/$(date +%F).json"

🔧 Customization

Add Custom Categories

-- Extend pattern categories
INSERT INTO learned_patterns (pattern_category, trigger_conditions, symptom_description, solution_template)
VALUES ('security', '{"contains": ["password", "auth"]}', 'Insecure authentication', 'Use bcrypt for password hashing');

Create Custom Queries

-- Find patterns in specific scenarios
SELECT scenario, COUNT(*) as frequency, AVG(human_rating) as avg_rating
FROM feedback_sessions 
WHERE scenario LIKE '%api%'
GROUP BY scenario
ORDER BY frequency DESC;

Batch Import

# Import from CSV
sqlite3 metrics.sqlite ".mode csv" ".import feedback_export.csv feedback_sessions"

📁 File Structure

learning-system/
├── README.md              # This file
├── schema.sql             # Database initialization
├── queries.sql            # Analysis queries
├── patterns.md            # Human-readable patterns
├── evolution.log          # Guideline change history
├── reports/               # Generated insights (gitignored)
│   ├── daily/
│   ├── weekly/
│   └── monthly/
├── scripts/               # Helper utilities
│   ├── add_feedback.sh
│   ├── daily_analysis.sh
│   └── export_patterns.py
└── examples/              # Usage examples
    ├── web_development/
    ├── data_science/
    └── mobile_app/

🔒 Privacy & Security

• Local Only: SQLite database stays on your machine (add metrics.sqlite to .gitignore)
• Sensitive Data: Avoid storing secrets in feedback_text or correction_code
• Sharing: Only commit human-readable summaries (patterns.md, reports/*.json)

🚧 Roadmap

v1.0 (Current)

• Core feedback collection
• Pattern extraction
• Priority ranking
• Daily analytics

v1.1 (Next)

• Web dashboard for visualization
• Automated pattern suggestions
• Integration with popular IDEs
• Slack/Discord notifications

v2.0 (Future)

• Multi-project aggregation
• Team collaboration features
• Machine learning pattern detection
• API for external tools

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Test with your own projects
4. Submit pull request with examples

Adding Examples

We welcome usage examples from different domains:

• Add to examples/<domain>/
• Include sample data and queries
• Document domain-specific patterns

📜 License

MIT License - see LICENSE file for details.

🆘 Support

• Issues: GitHub Issues for bugs and feature requests
• Discussions: GitHub Discussions for usage questions
• Documentation: Wiki for advanced usage patterns

---

Transform your AI assistance from reactive to proactive. Start learning from feedback today!