A reusable API performance testing framework built with Grafana k6 and JavaScript, using the public QuickPizza application.
The project demonstrates practical performance-testing engineering concepts including end-to-end API workflows, authentication, custom metrics, thresholds, configurable load profiles, negative API testing, reusable utilities, environment configuration, scenario-based execution, structured test organization, automated result analysis, baseline comparison, performance regression detection, and CI/CD quality gates.
Project status: Framework Development Complete — Iteration 25.13 signed off.
The framework has completed its planned development lifecycle, including automated performance analysis, version-controlled baseline comparison, regression detection, GitHub Actions CI/CD, and CI performance quality-gate validation.
This project is a hands-on learning and portfolio project focused on progressively building a maintainable k6 API performance-testing framework, from test execution and metrics collection through automated analysis, regression detection, and CI/CD validation.
- k6 API performance testing
- End-to-end API workflow
- User registration
- Authentication handling
- Rating API operations
- Negative API scenarios
- Reusable API functions
- Centralized environment configuration
- Runtime
BASE_URLconfiguration - Runtime
PASSWORDconfiguration - Configurable execution mode
- Configurable load profile
- Scenario-based execution
- Randomized test data
- Custom Trend metrics
- Custom Counter metrics
- Custom Rate metrics
- Scenario execution metrics
- Performance thresholds
- Group-level validation
- Endpoint-level thresholds
- Business-level success metrics
- Native k6 JSON result reporting
- Automated performance result analysis
- Scenario-aware analyzer thresholds
- Version-controlled smoke baseline
- Automated baseline comparison
- Performance regression detection
- CI performance quality gate
- GitHub Actions CI/CD
- Automated smoke performance execution
- Structured test organization
- Incremental framework development
| Technology | Purpose |
|---|---|
| Grafana k6 | API performance testing |
| JavaScript | Test implementation |
| QuickPizza API | Application under test |
| Node.js | Report analysis and comparison utilities |
| Git | Version control |
| GitHub | Source control and portfolio |
| GitHub Actions | CI/CD execution |
| VS Code / GitHub Codespaces | Development environment |
The purpose of this project is to build a maintainable API performance-testing framework rather than a collection of standalone k6 scripts.
The framework progressively demonstrates:
- API test implementation
- Reusable API abstractions
- Centralized configuration
- Test-data management
- Custom performance metrics
- Scenario configuration
- Performance thresholds
- Negative API testing
- Native JSON result generation
- Automated performance-result analysis
- Version-controlled performance baselines
- Regression detection
- CI/CD quality gates
QuickPizza API
│
▼
┌───────────────────┐
│ k6 Test Layer │
│ quickPizzaE2E.js │
└─────────┬─────────┘
│
┌───────────────┼────────────────┐
▼ ▼ ▼
Positive E2E Negative API Custom Metrics
│ │ │
└───────────────┼────────────────┘
▼
k6 Execution
│
▼
Native JSON Result
│
▼
analyzeReport.js
│
▼
Summary JSON
│
┌─────────┴─────────┐
▼ ▼
Thresholds Baseline Summary
│ │
└─────────┬─────────┘
▼
compareReports.js
│
▼
Regression Detection
│
┌──────┴──────┐
▼ ▼
PASS FAIL
GitHub Actions CI
│
▼
Automated Smoke Test
│
▼
Analyzer + Comparison
│
▼
Quality Gate
quickpizza-k6-performance-framework/
│
├── analyzer/
│ ├── analyzeReport.js
│ ├── thresholds.js
│ └── compareReports.js
│
├── api/
│ ├── userApi.js
│ └── ratingApi.js
│
├── config/
│ ├── env.js
│ ├── loadProfile.js
│ ├── scenarios.js
│ ├── thresholds.js
│ ├── negativeThresholds.js
│ └── smokeThresholds.js
│
├── data/
│ └── testData.js
│
├── utils/
│ ├── helpers.js
│ ├── metrics.js
│ └── request.js
│
├── tests/
│ ├── quickPizzaE2E.js
│ └── negative/
│ └── ratingNegativeTest.js
│
├── reports/
│ ├── .gitkeep
│ ├── smoke/
│ ├── load/
│ ├── negative/
│ └── baselines/
│ └── smoke/
│ └── smoke-baseline-summary.json
│
├── docs/
│ ├── analyzer.md
│ ├── performance-report-analysis.md
│ └── performance-comparison.md
│
├── .github/
│ └── workflows/
│ └── quickpizza-performance.yml
│
├── .gitignore
└── README.md
Generated k6 result files are excluded from Git, while the smoke baseline summary is intentionally version-controlled.
The main positive workflow validates a complete QuickPizza API transaction:
User Registration
↓
Authentication
↓
Create Order
↓
Get Order
↓
List Orders
↓
Verify Order
↓
Delete Order
↓
Cleanup
The workflow uses reusable API functions and custom metrics to capture both technical and business-level performance information.
Authentication is handled through reusable API utilities.
The framework supports:
- User registration
- Login
- Authentication token handling
- Authenticated API requests
- Runtime password configuration
Credentials are supplied through environment variables rather than hard-coded values.
API operations are separated from test-flow logic.
Responsible for user-related operations such as:
- Registration
- Login
Responsible for QuickPizza API operations such as:
- Creating orders
- Retrieving orders
- Listing orders
- Verifying orders
- Deleting orders
This separation improves reuse and keeps the test scenarios focused on business flows.
Negative API scenarios are maintained separately from the positive E2E workflow.
Current negative testing includes validation of API behavior for invalid request conditions.
Example:
Negative API Scenario
↓
Invalid Request
↓
API Response
↓
Status Validation
↓
Response Validation
↓
Performance Measurement
Negative tests are intentionally handled separately because their expected behavior and thresholds differ from positive performance scenarios.
The framework uses custom k6 metrics to measure business and technical behavior.
Used for measuring transaction duration and other continuous performance values.
Used to count successful business operations.
Used to measure reliability indicators such as:
- Login success rate
- Failure rate
The framework also tracks scenario execution information to make test results easier to analyze.
Performance thresholds are centralized rather than being distributed across individual tests.
The framework supports thresholds for:
- HTTP request duration
- HTTP request failure rate
- Check success rate
- Transaction duration
- Login success rate
- Successful business operations
- Endpoint-specific response times
- Group-level performance
Example:
HTTP request p95
↓
Threshold evaluation
↓
PASS / FAIL
Different scenarios have different performance expectations.
The framework currently distinguishes between:
Used for lightweight CI validation.
Smoke thresholds focus on stable indicators such as:
- HTTP response time
- HTTP failure rate
- Check success rate
- Transaction time
- Login success rate
Used for broader performance validation.
Load thresholds additionally include business-volume validation such as successful orders.
Negative scenarios use separate validation logic and do not use the positive-test threshold set.
Load behavior is configurable through:
config/loadProfile.js
This allows the framework to separate:
- Test logic
- Load configuration
- Scenario configuration
- Threshold configuration
The framework can therefore evolve from lightweight validation to longer performance runs without rewriting the test workflow.
Scenario definitions are centralized in:
config/scenarios.js
The framework currently supports scenario-based execution including:
- Smoke testing
- Load testing
- Negative testing
The selected scenario controls the appropriate execution configuration and threshold strategy.
Runtime environment configuration is centralized in:
config/env.js
The framework supports environment variables such as:
BASE_URL
PASSWORD
TEST_SCENARIO
Example:
$env:BASE_URL="https://quickpizza.grafana.com"
$env:PASSWORD="your-password"
$env:TEST_SCENARIO="smoke_test"This allows the same framework to be executed against different environments without modifying the test source code.
Test data is centralized in:
data/testData.js
This keeps test data separate from test-flow logic and makes future data expansion easier.
$env:TEST_SCENARIO="smoke_test"
k6 run --insecure-skip-tls-verify tests/quickPizzaE2E.js$env:TEST_SCENARIO="load_test"
k6 run --insecure-skip-tls-verify tests/quickPizzaE2E.jsk6 run --insecure-skip-tls-verify tests/negative/ratingNegativeTest.jsk6 native JSON output is used as the raw performance-test result.
Example:
k6 run --insecure-skip-tls-verify `
--out json=reports/smoke/smoke-result.json `
tests/quickPizzaE2E.jsThe resulting JSON is then processed by the framework analyzer.
k6 JSONL Result
↓
analyzeReport.js
↓
Structured Summary JSON
The raw k6 result is intentionally treated as the execution-level data source, while the summary JSON provides a stable structure for analysis and comparison.
The framework includes an automated report analyzer:
analyzer/analyzeReport.js
Its responsibilities include:
- Reading native k6 JSON results
- Aggregating performance metrics
- Calculating summary statistics
- Extracting reliability information
- Extracting endpoint metrics
- Extracting business metrics
- Evaluating scenario-specific thresholds
- Producing structured summary JSON
Execution:
node analyzer/analyzeReport.js reports/smoke/smoke-result.jsonOutput:
reports/smoke/smoke-summary.json
Analyzer thresholds are centralized in:
analyzer/thresholds.js
The analyzer selects thresholds according to the executed scenario.
Scenario
│
├── smoke_test → smoke thresholds
│
├── load_test → load thresholds
│
└── negative_test → negative analysis
This prevents load-specific business-volume thresholds from incorrectly failing lightweight smoke tests.
The framework compares the current performance summary against a version-controlled baseline.
Comparison utility:
analyzer/compareReports.js
Execution:
node analyzer/compareReports.js `
reports/baselines/smoke/smoke-baseline-summary.json `
reports/smoke/smoke-summary.jsonThe comparison evaluates:
- Performance metrics
- Reliability metrics
- Endpoint metrics
- Business metrics
- Scenario compatibility
The framework uses a 5% tolerance for automated regression detection.
Version-Controlled Baseline
↓
Current Summary
↓
Metric Comparison
↓
5% Tolerance
↓
┌───────┴────────┐
↓ ↓
No Regression Regression
↓ ↓
PASS FAIL
The comparison utility evaluates performance, reliability, endpoint and business metrics and classifies results as:
IMPROVEMENTREGRESSIONNO_CHANGENOT_COMPARABLENOT_AVAILABLE
The CI regression gate focuses on relatively stable request-level and reliability measurements, including:
- HTTP request duration
- HTTP request failure rate
- Check success rate
- Login success rate
- Endpoint-level performance metrics
transaction_time and iteration_duration remain part of performance reporting and threshold validation but are excluded from the automated baseline regression gate because smoke execution provides very few samples and these end-to-end measurements can vary between individual runs.
The comparison utility returns a non-zero exit code when a regression within the configured regression-gate metrics is detected, allowing CI/CD to enforce the performance quality gate.
The smoke baseline is stored in:
reports/baselines/smoke/smoke-baseline-summary.json
Only the structured summary is version-controlled.
Generated raw performance results remain excluded from Git.
This provides a lightweight approach to performance regression tracking without requiring a database or external performance platform.
The baseline is intentionally not overwritten automatically after every CI execution. Updating the baseline is a deliberate repository change.
The framework includes:
.github/workflows/quickpizza-performance.yml
The CI pipeline performs:
Git Push / Pull Request / Manual Run
↓
GitHub Actions Runner
↓
Checkout Repository
↓
Install k6
↓
Run QuickPizza Smoke Test
↓
Generate JSON Result
↓
Analyze Performance Result
↓
Generate Summary JSON
↓
Compare Against Baseline
↓
Regression Detection
↓
Quality Gate
The workflow uploads the generated smoke-test artifacts for inspection.
The CI pipeline fails when the smoke performance comparison detects a regression.
This makes performance testing part of the CI validation process rather than an isolated manual activity.
The quality-gate flow is:
Smoke Test
↓
Analyzer
↓
Summary
↓
Baseline Comparison
↓
Regression Detection
↓
┌───────────────┐
│ │
PASS FAIL
│ │
CI Continues CI Fails
The automated regression gate intentionally excludes transaction_time and iteration_duration because these end-to-end measurements are sensitive to the small sample size of smoke executions.
The framework captures multiple categories of metrics.
- Request duration
- Request failure rate
- Request count
- Check success rate
- Login success rate
- HTTP failure rate
- Transaction duration
- Scenario execution metrics
Metrics can be analyzed for individual operations such as:
register
login
create_order
get_order
list_orders
verify_order
delete_order
Examples include:
successful_orders
Historical internal metric and transaction names are intentionally preserved for framework consistency.
The framework validates both functional and performance behavior.
Validation includes:
- HTTP status checks
- Response checks
- Business checks
- Authentication validation
- Endpoint performance
- Transaction performance
- Scenario execution
- Threshold evaluation
- Negative API behavior
- Regression comparison
| Component | Responsibility |
|---|---|
api/ |
Reusable API operations |
config/ |
Runtime, scenario, load and threshold configuration |
data/ |
Centralized test data |
utils/ |
Shared helpers, metrics and request utilities |
tests/ |
Performance and negative test scenarios |
analyzer/ |
Result analysis and comparison |
reports/ |
Generated reports and version-controlled baselines |
docs/ |
Framework documentation |
.github/workflows/ |
CI/CD automation |
The framework follows several maintainability principles.
API operations, configuration, test data, utilities, analysis and test execution are separated.
Common operations are implemented as reusable functions.
Environment variables, scenarios, thresholds and load profiles are centralized.
Smoke, load and negative scenarios use appropriate execution and validation strategies.
Performance result analysis and regression detection are automated.
Smoke performance validation is integrated into GitHub Actions.
Performance regression detection uses a deliberate, version-controlled baseline rather than automatically changing the expected performance after every run.
| Iteration | Description | Status |
|---|---|---|
| 1 | Initialize k6 performance framework | ✅ |
| 2 | Centralize environment configuration | ✅ |
| 3 | Add reusable test data utilities | ✅ |
| 4 | Centralize custom k6 metrics | ✅ |
| 5 | Extract user API operations | ✅ |
| 6 | Extract rating API operations | ✅ |
| 7 | Centralize API request headers | ✅ |
| 8 | Add configurable load profile | ✅ |
| 9 | Add configurable performance scenarios | ✅ |
| 10 | Add smoke test scenario | ✅ |
| 11 | Validate performance test scenario | ✅ |
| 12 | Centralize performance thresholds | ✅ |
| 13 | Centralize test data | ✅ |
| 14 | Add negative API scenarios | ✅ |
| 15 | Add scenario execution metrics | ✅ |
| 16 | Add scenario tags to transaction metrics | ✅ |
| 17 | Expand negative API coverage | ✅ |
| 18 | Centralize negative test data | ✅ |
| 19 | Externalize environment configuration | ✅ |
| 20 | Framework validation and cleanup | ✅ |
| 21 | Configure performance report storage | ✅ |
| 22 | Add performance result analysis strategy | ✅ |
| 23 | Add automated k6 report analyzer | ✅ |
| 24 | Add performance result comparison and regression detection | ✅ |
| 25 | Add GitHub Actions CI/CD and performance quality gate | ✅ |
| 25.13 | Final validation, regression-gate refinement, CI validation and framework sign-off | ✅ |
The planned framework development is complete as of Iteration 25.13.
| Capability | Status |
|---|---|
| k6 API Testing | ✅ |
| QuickPizza E2E Workflow | ✅ |
| User Registration | ✅ |
| Authentication | ✅ |
| Rating API Operations | ✅ |
| Negative API Testing | ✅ |
| API Utilities | ✅ |
| Centralized Configuration | ✅ |
| Runtime Environment Configuration | ✅ |
| Custom Trend / Counter / Rate Metrics | ✅ |
| Scenario Execution Metrics | ✅ |
| Performance Thresholds | ✅ |
| Negative-Test Validation | ✅ |
| Configurable Load Profile | ✅ |
| Smoke / Load / Negative Scenarios | ✅ |
| Scenario Validation | ✅ |
| Native JSON Reporting | ✅ |
| Report Storage Structure | ✅ |
| Automated Result Analysis | ✅ |
| Scenario-Aware Analyzer Thresholds | ✅ |
| Version-Controlled Smoke Baseline | ✅ |
| Baseline Comparison | ✅ |
| Performance Regression Detection | ✅ |
| GitHub Actions CI/CD | ✅ |
| Automated Smoke Performance Execution | ✅ |
| CI Performance Quality Gate | ✅ |
| HTML Performance Reporting | 🔮 Future Enhancement |
| Historical Performance Trend Tracking | 🔮 Future Enhancement |
| Result Visualization | 🔮 Future Enhancement |
| Scheduled Heavy Performance Runs | 🔮 Future Enhancement |
The framework currently focuses on API performance testing using k6.
The primary goals are:
- Response-time validation
- Reliability measurement
- Transaction performance
- Endpoint-level performance
- Business-level performance indicators
- Smoke performance validation
- Load-test execution
- Negative API performance behavior
- Regression detection
The completed framework provides the core performance-testing architecture, while additional reporting and advanced execution capabilities remain available as future enhancements.
QuickPizza is a public application used for learning and portfolio demonstration.
Performance results can vary depending on:
- Network conditions
- Server-side workload
- Test execution environment
- Geographic location
- Time of execution
- Public service availability
Therefore, individual test results should be interpreted as observations from a specific test execution rather than permanent performance characteristics of the application.
This project demonstrates practical experience with:
- k6 API performance testing
- JavaScript-based performance automation
- API workflow design
- Authentication handling
- Test-data management
- Custom k6 metrics
- Performance thresholds
- Load profiles
- Scenario configuration
- Negative API testing
- JSON result processing
- Performance-result analysis
- Baseline comparison
- Regression detection
- CI/CD integration
- Performance quality gates
- Framework architecture
- Maintainable test automation design
The following improvements are intentionally kept as future work and are outside the completed Iteration 25 development scope:
- Additional QuickPizza API workflows
- Additional load profiles
- Expanded stress-testing scenarios
- Soak-testing scenarios
- HTML performance reports
- Improved result visualization
- Performance trend tracking
- Scheduled performance executions
- Additional CI performance scenarios
- Enhanced historical baseline management
These enhancements can be added in future versions as the portfolio project evolves.
Arindam Chowdhury
QA Automation Engineer / SDET
GitHub:
https://github.com/arindam0111
The goal of this project is to demonstrate how a maintainable API performance-testing framework can evolve from basic k6 test scripts into an engineering-oriented solution with:
Reusable API Layer
↓
Configurable Test Scenarios
↓
Custom Performance Metrics
↓
Performance Thresholds
↓
Native JSON Results
↓
Automated Result Analysis
↓
Version-Controlled Baseline
↓
Regression Detection
↓
CI/CD Quality Gate
This project is designed as a practical portfolio demonstration of QA Automation, API Testing, Performance Testing, JavaScript, k6, CI/CD, and SDET engineering practices.