test(cli): add comprehensive tests for --skip parameter validation#4
test(cli): add comprehensive tests for --skip parameter validation#4wbisschoff13 merged 5 commits intomainfrom
Conversation
Task: TM-005 - Add CLI validation for skip parameter Notes: - Tests cover validation logic for negative values, zero, and positive integers - Tests verify decimal values are rejected with descriptive error messages - Tests confirm undefined skip parameter passes validation (optional) - Tests verify skip functionality correctly skips eligible tasks - Validates Commander.js string-to-number conversion handling - Includes .taskmaster/tasks/tasks.json with task #5 status updated to "done" Validation logic was implemented in commit 41b319a; this commit adds the corresponding test coverage as specified in the task's test strategy.
Summary of ChangesHello @wbisschoff13, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the robustness of the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request adds a comprehensive suite of integration tests for the --skip parameter, which is a great addition for ensuring the reliability of the CLI. The tests cover various validation and functionality scenarios. My review focuses on opportunities to strengthen the assertions in some of the new tests to make them more precise and to refactor the test structure for better maintainability by reducing code duplication. Overall, these are solid tests, and my suggestions aim to make them even more robust.
| it('should handle non-numeric skip values (Commander.js validation)', () => { | ||
| // Commander.js handles non-numeric values before our custom validation | ||
| // This test documents the current behavior | ||
| const { output } = runNext('--skip abc'); | ||
|
|
||
| // Commander.js may reject or accept this depending on version | ||
| // The important thing is that it doesn't crash | ||
| expect(output).toBeDefined(); | ||
| }); |
There was a problem hiding this comment.
The assertion in this test is too weak. It only checks that output is defined but doesn't verify the expected failure. The application's custom validation logic in next.command.ts should catch non-integer values and throw an error. The test should assert that the command fails with an exit code of 1 and that the output contains the expected error message for a more robust check.
it('should throw a descriptive error for non-numeric skip values', () => {
const { output, exitCode } = runNext('--skip abc');
expect(exitCode).toBe(1);
expect(output).toContain('Invalid skip count');
expect(output).toContain('non-negative integer');
});| describe('--skip parameter functionality', () => { | ||
| it('should skip the specified number of eligible tasks', () => { | ||
| const testData = createTasksFile({ | ||
| tasks: [ | ||
| createTask({ id: 1, title: 'First Task', status: 'pending' }), | ||
| createTask({ id: 2, title: 'Second Task', status: 'pending' }), | ||
| createTask({ id: 3, title: 'Third Task', status: 'pending' }) | ||
| ] | ||
| }); | ||
| writeTasks(testData); | ||
|
|
||
| const { output: firstOutput } = runNext(); | ||
| const firstTaskId = firstOutput.match(/Next Task: #(\d+)/)?.[1]; | ||
|
|
||
| const { output: secondOutput } = runNext('--skip 1'); | ||
| const secondTaskId = secondOutput.match(/Next Task: #(\d+)/)?.[1]; | ||
|
|
||
| // The task returned with --skip 1 should be different from the first task | ||
| expect(firstTaskId).toBeDefined(); | ||
| expect(secondTaskId).toBeDefined(); | ||
| expect(secondTaskId).not.toBe(firstTaskId); | ||
| }); | ||
|
|
||
| it('should skip multiple tasks when specified', () => { | ||
| const testData = createTasksFile({ | ||
| tasks: [ | ||
| createTask({ id: 1, title: 'Task One', status: 'pending' }), | ||
| createTask({ id: 2, title: 'Task Two', status: 'pending' }), | ||
| createTask({ id: 3, title: 'Task Three', status: 'pending' }) | ||
| ] | ||
| }); | ||
| writeTasks(testData); | ||
|
|
||
| const { output } = runNext('--skip 2'); | ||
|
|
||
| expect(output).toContain('Task'); | ||
| // Should return one of the tasks (implementation decides which) | ||
| }); | ||
| }); |
There was a problem hiding this comment.
The tests for --skip functionality can be improved in a couple of ways:
- Reduce Duplication: The test data setup is repeated in both tests. This can be extracted into a
beforeEachhook to keep the tests DRY and improve maintainability. - Strengthen Assertions: The current assertions are not specific enough. For instance, one test only checks that the skipped task is "different", and another just checks for the word "Task" in the output. Since the task selection logic is deterministic (sorted by ID in this case), the tests should assert that the exact correct task is returned for each skip value.
Here's a suggested refactoring that addresses both points by using a beforeEach hook and adding more specific assertions, splitting the logic into clearer, more focused tests.
describe('--skip parameter functionality', () => {
beforeEach(() => {
const testData = createTasksFile({
tasks: [
createTask({ id: 1, title: 'First Task', status: 'pending' }),
createTask({ id: 2, title: 'Second Task', status: 'pending' }),
createTask({ id: 3, title: 'Third Task', status: 'pending' })
]
});
writeTasks(testData);
});
it('should return the first task when no skip is provided', () => {
const { output } = runNext();
expect(output).toContain('Next Task: #1');
});
it('should skip one task and return the second task', () => {
const { output } = runNext('--skip 1');
expect(output).toContain('Next Task: #2');
});
it('should skip two tasks and return the third task', () => {
const { output } = runNext('--skip 2');
expect(output).toContain('Next Task: #3');
});
});Applied PR review feedback to improve test quality: - Strengthen non-numeric skip test to verify proper error handling * Add exitCode assertion (should be 1) * Verify error message contains "Invalid skip count" * Verify error message contains "non-negative integer" - Refactor --skip functionality tests to reduce duplication * Extract test data setup into beforeEach hook * Add specific assertions verifying exact task IDs returned * Split into clearer, focused tests for each skip value All 19 tests pass successfully.
* feat(tasks): add skipCount parameter to getNextTask method Task: TM-001 - Add skipCount parameter to service layer Subtasks: - TM-001.1: Update method signature with TypeScript types - TM-001.2: Implement skip indexing logic with array bounds checking - TM-001.3: Add input validation for skipCount parameter - TM-001.4: Ensure backward compatibility with existing calls - TM-001.5: Add comprehensive inline documentation Notes: - Added optional skipCount parameter to getNextTask(tag?: string, skipCount?: number) - Implemented skip indexing: returns task at index skipCount from eligible tasks - Validates skipCount must be non-negative integer; throws TaskMasterError if invalid - Returns null if skipCount exceeds eligible tasks length - Backward compatible: default skipCount=0 preserves existing behavior - Applied to both subtask and top-level task selection logic - Updated JSDoc with usage examples and parameter descriptions - Includes .taskmaster/tasks.json with task completion status Files modified: - packages/tm-core/src/modules/tasks/services/task-service.ts * docs(pr): add PR description for task/1-add-skipcount-parameter * docs(pr): rename description to PR #1 * fix(tasks): correct skip count propagation in getNextTask Fix logical flaw in skipCount parameter handling where falling through from subtasks to top-level tasks didn't adjust the skip value. This caused incorrect number of top-level tasks to be skipped. Changes: - Change skip variable from const to let for mutability - Decrement skip by candidateSubtasks.length when falling through Fixes bug where skipCount=2 with 2 available subtasks would incorrectly skip the first 2 top-level tasks instead of returning the first one. feat(tasks): add skipCount parameter to getNext domain method (#2) * feat(tasks): add skipCount parameter to getNext domain method Task: TM-2 - Update domain facade with skipCount parameter Added optional skipCount parameter to TasksDomain.getNext() method to support skipping eligible tasks when retrieving the next available task. This enables use cases like retrying task selection after skipping previously attempted tasks. Implementation: - Updated getNext method signature to accept optional skipCount parameter - Added comprehensive JSDoc documentation with usage examples - Parameter pass-through to taskService.getNextTask() maintaining architecture - Full backward compatibility (undefined default preserves existing behavior) - Added 7 unit tests covering all skipCount scenarios and edge cases Test coverage includes: - Parameter pass-through verification - Backward compatibility (undefined skipCount) - skipCount=0 (first eligible task) - skipCount>0 (skip N tasks) - Error propagation from service layer - Return value handling (task and null cases) - Type safety verification Files modified: - packages/tm-core/src/modules/tasks/tasks-domain.ts:152-172 - packages/tm-core/src/modules/tasks/tasks-domain.spec.ts (new file) - .taskmaster/tasks/tasks.json (task status updated to done) * docs(tasks): clarify skipCount fallthrough logic in getNextTask Task: TM-002 - Update domain facade with skipCount parameter Notes: - Improved comment clarity for skipCount parameter fallthrough behavior - Clarifies that skipCount is relative to combined eligible pool of subtasks and tasks - Helps future developers understand the two-stage task selection logic - No functional changes - documentation improvement only File: packages/tm-core/src/modules/tasks/services/task-service.ts:436-440 * docs(pr): add PR description for task/2-domain-skipcount-parameter * docs(pr): rename description to PR #2 * test(tasks): improve type safety in domain test mocks Replace inline mock object creation with type-safe helper function. Adds createMockTaskService() factory to reduce 'as any' casting and improve test maintainability without adding dependencies. Addresses PR review feedback on mock type safety. feat(cli): add --skip parameter with integer validation (#3) * feat(cli): add --skip parameter with integer validation Enable parallel workflows by allowing users to skip eligible tasks. Validates that skip count is a non-negative integer to prevent invalid input like --skip 1.5 * docs(pr): add PR description for task/3-cli-skip-param Generated comprehensive PR description for the --skip parameter feature, including technical implementation details, testing steps, and risk assessment. * docs(pr): rename description to PR #3 * fix(cli): handle Commander string option for skip parameter Commander passes CLI options as strings, not numbers. The previous validation logic incorrectly used Number.isInteger() on a string, which always returns false and rejects valid inputs like "1". Changes: - Update skip type to number | string to accept Commander input - Parse string to number before validation - Assign numeric value back to options.skip for use in getNextTask - Add type assertion when passing to tm-core (guaranteed number) Fixes validation failure for: task-master next --skip 1 test(cli): add comprehensive tests for --skip parameter validation (#4) * chore(task-master): mark task #4 as done after CLI implementation Task: TM-004 - Add --skip CLI option with Commander.js Notes: - Task implementation completed in commits 85a6066, 3cef35f, and 41b319a - Updates task status from "pending" to "done" - Increments completedCount from 3 to 4 - Includes .taskmaster/tasks.json with task status update * test(cli): add comprehensive tests for --skip parameter validation Task: TM-005 - Add CLI validation for skip parameter Notes: - Tests cover validation logic for negative values, zero, and positive integers - Tests verify decimal values are rejected with descriptive error messages - Tests confirm undefined skip parameter passes validation (optional) - Tests verify skip functionality correctly skips eligible tasks - Validates Commander.js string-to-number conversion handling - Includes .taskmaster/tasks/tasks.json with task #5 status updated to "done" Validation logic was implemented in commit 41b319a; this commit adds the corresponding test coverage as specified in the task's test strategy. * docs(pr): add PR description for --skip parameter test coverage * docs(pr): rename description to PR #4 * test(cli): strengthen --skip parameter test assertions Applied PR review feedback to improve test quality: - Strengthen non-numeric skip test to verify proper error handling * Add exitCode assertion (should be 1) * Verify error message contains "Invalid skip count" * Verify error message contains "non-negative integer" - Refactor --skip functionality tests to reduce duplication * Extract test data setup into beforeEach hook * Add specific assertions verifying exact task IDs returned * Split into clearer, focused tests for each skip value All 19 tests pass successfully. refactor(cli): improve error handling and code reuse for --skip parameter (#5) * chore(task-master): mark task #6 as done - CLI skip parameter integration complete Task: TM-006 - Update CLI getNextTask call with skip parameter Notes: - Task #6 implementation was completed in commit 6dd6ab0 (test(cli): add comprehensive tests for --skip parameter validation) - The getNextTask method already passes skip parameter to tmCore.tasks.getNext() - This commit updates Task-Master state to reflect completion - Skip parameter is properly validated and passed through from CLI options to domain layer - All tests passing and backward compatibility maintained * refactor(skip): improve error handling, naming, and code reuse Code quality improvements from code review: - Use TaskMasterError consistently in CLI validation - Improve skip error messages with quoted values and examples - Rename 'skip' to 'remainingSkip' for clearer fallthrough logic - Extract getSkipIndex() utility to eliminate duplicate bounds checking - Fix TypeScript compilation error in test mock * docs(pr): add PR description for task/6-cli-next-skip-param Generated comprehensive PR description covering: - Refactoring of skip parameter error handling - Code reuse improvements with getSkipIndex() helper - Enhanced validation error messages - Test verification results Risk assessment: Low risk internal refactoring * docs(pr): rename description to PR #5 * chore: address PR review feedback Applied 2 review comments from PR #5: - Add quotes around invalid format value in error message for consistency - Make getSkipIndex method static since it doesn't use instance state - Update method calls to use static syntax * refactor(skip): simplify getSkipIndex with nullish coalescing Applied PR review feedback from PR #5: - Simplified getSkipIndex helper using nullish coalescing operator (??) - Method already static, no changes needed there - Format validation error already correctly quotes invalid value The refactored implementation is more concise while maintaining the same behavior: returns the item at the skip index or null if out of bounds. feat(cli): add skip-exceeded detection with user-friendly messaging (#6) * feat(cli): add skip-exceeded detection with user-friendly messaging Task: TM-007 - Update skip logic in display text output Implementation: - Added skipValue and availableTaskCount to NextTaskResult interface - Calculate eligible tasks (pending/deferred with no unmet dependencies) - Detect when --skip exceeds available eligible tasks - Display clear message with actual skip value and available count - Provide helpful tip to get last available task Notes: - Maintains existing behavior for non-skip null task cases - Preserves JSON output format (only affects text display) - Uses existing chalk formatting patterns for consistency - Edge cases: skip=0 returns first task, skip>=count shows message - Includes .taskmaster/tasks.json with task status updated to "done" Files changed: - apps/cli/src/commands/next.command.ts: Display logic enhancements - .taskmaster/tasks/tasks.json: Task 7 marked complete * refactor(cli): extract eligibility logic to improve testability Extract task eligibility calculation into separate helper methods: - Add countEligibleTasks() to count tasks available for work - Add areAllDependenciesDone() to check dependency completion - Improve type assertion documentation for clarity This refactoring improves testability, reusability, and maintainability by separating concerns. The main getNextTask() method is now more readable and the eligibility logic can be independently tested. * docs(pr): add PR description for task/7-update-skip-display-logic * docs(pr): rename description to PR #6 * perf(cli): optimize dependency lookup with Map for O(N) complexity Use Map instead of Array.find() for dependency lookups in eligibility calculation. Reduces time complexity from O(N²) to O(N) and fixes correctness issue where missing dependencies were silently ignored. Addresses PR #6 review feedback on performance optimization. fix(tasks): ensure skip parameter treats parent+subtasks as single unit (#7) * fix(tasks): ensure skip parameter treats parent+subtasks as single unit Task: TM-009 - Write comprehensive unit tests for skip logic Fixed critical bug where parent tasks with eligible subtasks were included in the top-level eligible tasks array, causing incorrect skip counting. Now the entire parent task hierarchy (parent + all eligible subtasks) is treated as a single logical unit in the skip index. Changes: - Added logic to exclude parent tasks with eligible subtasks from eligibleTasks - Updated JSDoc with "Parent Task Hierarchy" section and concrete examples - Added 33 comprehensive unit tests covering all skip scenarios - Tests verify skip behavior with subtasks, dependencies, priorities, and boundaries Example behavior: - Task 1 has subtasks 1.1, 1.2, 1.3; task 2 exists - skip=0 → 1.1, skip=1 → 1.2, skip=2 → 1.3, skip=3 → task 2 - Previously skip=3 would incorrectly return task 1 (parent) instead of task 2 Test coverage: - Basic skip indexing (0, 1, N) - Boundary conditions (exceeds available, equals count, negative) - Priority sorting with skip - Duplicate priority tiebreakers (ID, dependency count) - Dependency resolution with skip - Subtasks from in-progress parents (critical fix) - Backward compatibility (undefined defaults to 0) - Tag filtering and status filtering - Performance with large task sets (1000+ tasks) All 33 tests pass successfully. * docs(pr): add PR description and fix test type errors Add comprehensive PR description for the skip logic bug fix that ensures parent task hierarchies are treated as a single unit in the skip index. Also fix TypeScript type errors in the new test file by adding all required IStorage interface methods and completing Subtask interface fields. * docs(pr): rename description to PR #7 * refactor(tasks): address PR review feedback - code quality improvements Context: - Task: PR #7 Review Feedback - Source: #7 Notes: - Extracted isSubtaskEligible() helper to eliminate duplication in subtask eligibility logic - Refactored 2 error-handling tests to use idiomatic vitest rejects.toHaveProperty matcher - Changes reduce code by 7 lines net while improving maintainability - All 1,483 tests pass test(cli): add JSON output tests for skip parameter (#8) * test(cli): add JSON output tests for skip parameter Task: TM-008 - Add skip parameter tests for JSON output Subtasks: None (single logical unit) Notes: - Added 21 comprehensive integration tests for JSON output with --skip - Enhanced parseJsonOutput helper to extract JSON from mixed output - Tests cover edge cases: skip=0, skip exceeds available, empty task lists - Verified compatibility with --tag, --silent, and --format options - Validated backward compatibility with existing JSON structure - All 40 tests passing (19 existing + 21 new) - No code changes needed - existing JSON output was already correct - Includes .taskmaster/tasks.json with task status updated to "done" * docs(pr): add PR description for task/8-json-skip-tests * docs(pr): rename description to PR #8 * test(cli): refactor JSON tests to apply review feedback - Combine duplicate JSON structure validation tests using it.each - Improve skip=0 comparison to verify entire object, not just task ID feat(expand): add complexity threshold filtering to expand --all (#9) * feat(expand): add complexity threshold filtering to expand --all Add optional --threshold flag to filter tasks by minimum complexity score (1-10) before expansion. Requires complexity report from analyze-complexity command. Gracefully degrades when report is missing. Features: - MCP tool: threshold parameter in expand-all schema with range validation - CLI: --threshold / -t flag for expand command - Core: filter tasks by complexity score before expansion - Tests: 5 test cases covering filtering edge cases - UX: improved error messages guide users to run analyze-complexity - Tests: removed global fs mock, use local spyOn with cleanup Implementation details: - Validates threshold is between 1-10, warns and ignores if invalid - Logs before/after counts when threshold filtering is active - Provides actionable error message when complexity report is missing - Test isolation improved by removing global fs mock * docs(pr): add PR description for complexity threshold filtering feature * docs(pr): rename description to PR #9 * refactor(expand): optimize threshold filtering and improve test isolation Simplify null check using loose equality and optimize O(N*M) complexity to O(N+M) using Map for task analysis lookups. Add proper mock cleanup in tests to prevent state leakage between test runs. Changes: - Use != null instead of !== null && !== undefined for concise null check - Replace array.find() with Map.get() for O(1) lookup performance - Add spy mockRestore() calls to threshold filtering tests test(cli): fix integration tests for --skip parameter alignment (#10) * test(cli): fix integration tests for --skip parameter alignment Task: TM-010 - Write comprehensive integration tests for CLI Subtasks: - TM-010.1: Set up test infrastructure with temporary directories and test projects Notes: - Removed 8 incorrect tests that expected --tag to filter by task tags property - --tag parameter is for file section selection (legacy format), not task tags filtering - Fixed timeout syntax for parallel workflow tests (3 tests use 30s timeout) - Adjusted default vitest timeouts: root 10s→5s, cli 30s→10s - All 44 integration tests now passing - Includes .taskmaster/tasks/tasks.json with task status update * docs(pr): add PR description for task/10-cli-integration-tests * docs(pr): rename description to PR #10
Add comprehensive test coverage for the
--skipCLI parameter, ensuring robust validation and functionality for task skipping behavior. This test suite verifies that the CLI properly validates the skip parameter (rejecting negative and decimal values) and correctly skips the specified number of eligible tasks when retrieving the next task.Key changes
apps/cli/tests/integration/commands/next.command.test.tscovering:runNexttest helper to accept command arguments for flexible testingHow to test
Automated Tests
npm run turbo:typecheck-npm run test -w @tm/cli -- tests/integration/commands/next.command.test.ts- All 17 tests pass (✓)Manual Testing
Risk Assessment (Low)
This is a low-risk test-only addition with no production code changes:
The test suite provides confidence that the
--skipparameter implementation (added in previous tasks) works correctly and handles edge cases appropriately.Checklist