A comprehensive test planning and automation framework with structured operational modes and memory persistence.
This framework provides a structured approach to test planning, creation, execution, and reporting using a mode-based system that maintains context between sessions. It implements a memory bank system that preserves information across different operational phases.
graph LR
A[Start] --> B[ANALYSIS Mode]
B --> C[PLANNING Mode]
C --> D[CREATION Mode]
D --> E[EXECUTION Mode]
E --> F[REPORTING Mode]
F --> G[End/Iterate]
B -.-> MB[(Memory Bank)]
C -.-> MB
D -.-> MB
E -.-> MB
F -.-> MB
style A fill:#f9f,stroke:#333,stroke-width:2px
style G fill:#f9f,stroke:#333,stroke-width:2px
style MB fill:#bbf,stroke:#333,stroke-width:2px
graph TD
A[User Input] --> B{Mode Selection}
B -->|ANALYSIS| C[Analysis Mode]
B -->|PLANNING| D[Planning Mode]
B -->|CREATION| E[Creation Mode]
B -->|EXECUTION| F[Execution Mode]
B -->|REPORTING| G[Reporting Mode]
C --> H[Interactive Questioning]
H --> I[Generate Project Brief]
I --> J[Update Memory Bank]
D --> K[Load Project Context]
K --> L[Develop Test Strategy]
L --> M[Create Test Plan]
M --> J
E --> N[Load Test Plan]
N --> O[Build Test Framework]
O --> P[Create Test Scripts]
P --> J
F --> Q[Execute Tests]
Q --> R[Log Results]
R --> S[Track Defects]
S --> J
G --> T[Analyze Results]
T --> U[Generate Reports]
U --> V[Provide Recommendations]
V --> J
J --> W[Context Preserved]
W --> X[Ready for Next Mode]
style A fill:#f96,stroke:#333,stroke-width:2px
style J fill:#bbf,stroke:#333,stroke-width:2px
style W fill:#9f9,stroke:#333,stroke-width:2px
The framework operates in five distinct modes:
Interactive questioning flow to gather comprehensive information about the application under test. Produces a structured project brief document.
Develops detailed test plans based on the information gathered during Analysis. Creates test strategies, risk assessments, and prioritization.
Builds test scripts, frameworks, and other necessary artifacts based on the test plan.
Runs tests, logs results, monitors execution, and tracks defects.
Generates comprehensive reports with insights, metrics, and recommendations.
The framework maintains context using a structured memory bank:
- projectbrief.md - Core document containing all application and testing requirements
- activeContext.md - Current testing focus and contextual information
- tasks.md - Task tracking and next steps
- progress.md - Overall progress tracking and status reporting
Each mode follows a specific process flow defined in its corresponding process map:
- analysis-mode-map.md - Interactive questioning flow for gathering requirements
- planning-mode-map.md - Test planning process flow
- creation-mode-map.md - Test framework and script creation flow
- execution-mode-map.md - Test execution and monitoring flow
- reporting-mode-map.md - Result analysis and reporting flow
# Start with Analysis mode for a new project
> ANALYSIS
OK ANALYSIS
# The framework will begin the interactive questioning flow
What is the name of the application under test?
> MyWebApp
What is the primary purpose of the application?
> E-commerce platform for online retail sales
# Continue answering questions to build comprehensive project brief# The framework automatically loads existing context
> PLANNING
OK PLANNING
# Loads previous analysis and begins test planning
Loading project context from memory bank...
Project: MyWebApp
Application Type: Web Application
Ready to begin test planning.# Step 1: Analysis Mode
> ANALYSIS
OK ANALYSIS
# Interactive Q&A session builds project brief
Application Name: CustomerPortal
Type: Web Application
Technology: React, Node.js, PostgreSQL
Users: Internal staff, external customers
Critical Features: Login, payment processing, reporting
# Step 2: Planning Mode
> PLANNING
OK PLANNING
# Framework creates comprehensive test plan
Test Strategy: Risk-based approach
High Priority: Payment processing, user authentication
Medium Priority: Reporting features, UI validation
Low Priority: Administrative functions
# Step 3: Creation Mode
> CREATION
OK CREATION
# Framework guides test script development
Test Framework: Selenium WebDriver + Jest
Page Object Model implemented
Test data management strategy defined
CI/CD integration configured
# Step 4: Execution Mode
> EXECUTION
OK EXECUTION
# Test execution and monitoring
Test Suite: CustomerPortal_Regression_v1.2
Passed: 145/150 tests
Failed: 5 tests (payment validation issues)
Defects logged: 3 critical, 2 minor
# Step 5: Reporting Mode
> REPORTING
OK REPORTING
# Comprehensive analysis and recommendations
Test Coverage: 87% of critical paths
Defect Density: 0.3 defects per feature
Recommendation: Address payment validation before release# Load existing project and focus on specific planning area
> PLANNING
OK PLANNING
# Framework detects existing project brief
Project loaded: CustomerPortal
Previous analysis complete: Yes
Focus area: Integration testing strategy
# Specific planning activities
Integration points identified: 5 external APIs
Mock strategies defined for payment gateway
Test data requirements established
Environment setup documented# Monitor ongoing test execution
> EXECUTION
OK EXECUTION
# Real-time execution monitoring
Current Suite: Regression_Suite_v2.1
Progress: 67% complete (201/300 tests)
Duration: 2h 15m elapsed
Failed Tests: 12 (mostly UI timing issues)
# Defect tracking integration
New defects: 3
Critical defects: 1 (payment gateway timeout)
Defects resolved since last run: 5| Command | Description | Example Usage |
|---|---|---|
ANALYSIS |
Start/resume requirements gathering | Begin new project analysis |
PLANNING |
Enter test planning mode | Create test strategy and plans |
CREATION |
Build test frameworks and scripts | Develop automation assets |
EXECUTION |
Run tests and monitor results | Execute test suites |
REPORTING |
Generate reports and insights | Create stakeholder reports |
STATUS |
Check current project status | View progress across all modes |
CONTEXT |
Display current context | Show active project information |
HELP [MODE] |
Get mode-specific guidance | Show available commands |
- Adaptive Questioning - Intelligent questioning flow that adapts based on previous answers
- Structured Documentation - Comprehensive document generation with consistent formatting
- Context Preservation - Seamless transition between modes with full context preservation
- Risk-Based Approach - Prioritization based on business risk and technical complexity
- Comprehensive Coverage - Holistic approach covering all aspects of testing
- Modular Design - Extensible architecture for adding new capabilities
The framework implements a comprehensive versioning system to manage test plans across different releases and iterations:
[ProjectName]_TestPlan_v[Major].[Minor].[Patch]_[Environment]_[Date]
Examples:
- CustomerPortal_TestPlan_v2.1.0_Production_20240115
- PaymentAPI_TestPlan_v1.3.2_Staging_20240118
- MobileApp_TestPlan_v3.0.0_UAT_20240120
test-plans/
├── current/
│ ├── CustomerPortal_TestPlan_v2.1.0.md
│ ├── test-strategy-v2.1.md
│ └── risk-assessment-v2.1.md
├── archive/
│ ├── v2.0.x/
│ │ ├── CustomerPortal_TestPlan_v2.0.3.md
│ │ └── CustomerPortal_TestPlan_v2.0.2.md
│ └── v1.x.x/
│ ├── CustomerPortal_TestPlan_v1.2.1.md
│ └── CustomerPortal_TestPlan_v1.1.0.md
└── templates/
├── test-plan-template.md
└── risk-matrix-template.md
# Create new version
> PLANNING --version 2.1.0
Creating test plan version 2.1.0
Base version: 2.0.3
Changes: Added mobile testing scenarios
# Reference specific version
> EXECUTION --plan-version 2.0.3
Loading test plan version 2.0.3
Execution context: Production deployment
# Compare versions
> PLANNING --compare v2.0.3 v2.1.0
Comparing test plan versions:
Added: 15 mobile test cases
Modified: 8 API validation tests
Removed: 3 deprecated UI tests| Version Type | When to Use | Example | Retention |
|---|---|---|---|
| Major (x.0.0) | New product release, architecture changes | v3.0.0 | Indefinite |
| Minor (1.x.0) | New features, significant test additions | v1.5.0 | 2 years |
| Patch (1.1.x) | Bug fixes, small test modifications | v1.1.3 | 1 year |
| Hotfix | Critical production issues | v1.1.3-hotfix1 | 6 months |
# Development branch testing
> PLANNING --branch develop --version 2.2.0-dev
Test plan for development branch
Version: 2.2.0-dev
Status: In development
# Release candidate testing
> PLANNING --branch release/v2.2.0 --version 2.2.0-rc1
Test plan for release candidate
Version: 2.2.0-rc1
Status: Release testing
# Production release
> PLANNING --branch main --version 2.2.0
Test plan for production
Version: 2.2.0
Status: Production readyEach version maintains a detailed change log:
# Test Plan Change Log
## Version 2.1.0 (2024-01-15)
### Added
- Mobile responsiveness test scenarios (15 cases)
- API rate limiting validation tests (8 cases)
- Performance benchmarking suite
### Changed
- Updated user authentication test flows
- Enhanced security testing coverage
- Modified data validation test cases
### Removed
- Deprecated legacy browser tests (IE11)
- Obsolete Flash-based functionality tests
### Fixed
- Test data setup inconsistencies
- Environment configuration issues# Auto-increment version based on changes
> PLANNING --auto-version
Analyzing changes since v2.0.3...
Detected: 15 new test cases, 8 modifications
Recommended version: v2.1.0
Proceed? (Y/n)
# Merge version from branch
> PLANNING --merge-version feature/payment-integration
Merging test plan changes from feature branch
New test cases: Payment gateway integration (12 cases)
Updated version: v2.1.0qa-agent/
├── ai-docs/ # Generated documentation
│ ├── business-logic.md # Business logic documentation
│ └── technical-specs.md # Technical specifications
├── isolation_rules/ # Mode process definitions
│ └── visual-maps/
│ ├── analysis-mode-map.md # Analysis workflow
│ ├── planning-mode-map.md # Planning workflow
│ ├── creation-mode-map.md # Creation workflow
│ ├── execution-mode-map.md # Execution workflow
│ └── reporting-mode-map.md # Reporting workflow
├── memory_bank/ # Persistent context storage
│ ├── projectbrief.md # Application requirements
│ ├── activeContext.md # Current session state
│ ├── tasks.md # Task tracking
│ └── progress.md # Progress monitoring
├── test-plans/ # Versioned test plans
│ ├── current/ # Active test plans
│ │ ├── [Project]_TestPlan_v*.md
│ │ ├── test-strategy-v*.md
│ │ └── risk-assessment-v*.md
│ ├── archive/ # Historical versions
│ │ ├── v1.x.x/
│ │ ├── v2.x.x/
│ │ └── v3.x.x/
│ └── templates/ # Template files
│ ├── test-plan-template.md
│ └── risk-matrix-template.md
├── test-assets/ # Generated test artifacts
│ ├── frameworks/ # Test frameworks
│ ├── scripts/ # Test scripts
│ ├── data/ # Test data
│ └── reports/ # Execution reports
├── README.md # This documentation
└── qa-agent-entry.md # Framework entry point
The framework is designed to be extensible in several ways:
- New Modes - Additional operational modes can be added by creating new process maps
- Enhanced Questioning - The questioning flow can be expanded with new categories or questions
- Custom Reporting - The reporting mode can be extended with new visualization types
- Integration Capabilities - The framework can be integrated with specific testing tools
- Complete ANALYSIS thoroughly before proceeding to PLANNING mode
- Update tasks.md regularly to maintain accurate progress tracking
- Review and approve documents at the end of each mode transition
- Follow verification steps defined in each mode's process map
- Use semantic versioning for all test plan versions (Major.Minor.Patch)
- Archive old versions systematically to maintain history
- Document changes in detailed change logs for each version
- Tag releases with corresponding test plan versions
- Preserve context between sessions using the memory bank
- Update activeContext.md when changing focus areas
- Reference previous decisions when making new choices
- Maintain traceability from requirements through execution
- Validate completeness of each mode before transitioning
- Cross-reference test coverage with business requirements
- Monitor progress using the progress.md tracking system
- Review risk assessments regularly during project evolution
# Problem: Command not recognized
> ANALSIS
Error: Unknown command 'ANALSIS'
# Solution: Use correct mode names
> ANALYSIS
OK ANALYSIS# Problem: Previous work not found
> PLANNING
Warning: No project brief found
# Solution: Complete Analysis mode first
> ANALYSIS
OK ANALYSIS
# Complete the questioning flow, then proceed# Problem: Multiple projects in same directory
Error: Conflicting project contexts detected
# Solution: Use project-specific directories
mkdir project-a && cd project-a
> ANALYSIS --project "ProjectA"# Problem: Version already exists
> PLANNING --version 2.1.0
Error: Version 2.1.0 already exists
# Solution: Use next available version or force update
> PLANNING --version 2.1.1
> PLANNING --version 2.1.0 --force| Command | Purpose | Example |
|---|---|---|
HELP |
General framework help | Show all available commands |
HELP [MODE] |
Mode-specific guidance | HELP ANALYSIS |
STATUS --verbose |
Detailed project status | Show full context and progress |
CONTEXT --debug |
Debug context issues | Show memory bank details |