Skip to content

Comments

feat: Validate step input overrides against model schema (#244)#255

Merged
stack72 merged 1 commit intomainfrom
feat/validate-step-input-overrides
Feb 10, 2026
Merged

feat: Validate step input overrides against model schema (#244)#255
stack72 merged 1 commit intomainfrom
feat/validate-step-input-overrides

Conversation

@stack72
Copy link
Contributor

@stack72 stack72 commented Feb 10, 2026

Implements input override validation for step/CLI inputs that override model definition attributes (implicit inputs). This
ensures users get helpful error messages when providing invalid input keys or type mismatches.

Closes #244

Changes

New Files

  • src/domain/inputs/input_override_validation_service.ts - Validation service with Levenshtein-based typo detection
  • src/domain/inputs/input_override_validation_service_test.ts - 16 unit tests covering all validation scenarios
  • integration/input_override_validation_test.ts - Integration tests for CLI and workflow paths

Modified Files

  • src/domain/inputs/mod.ts - Export new service
  • src/cli/commands/model_method_run.ts - Add validation before applying overrides
  • src/domain/workflows/execution_service.ts - Add validation for workflow step inputs

Integration Test Fixes

Fixed several integration tests that broke due to:

  • Workflow filename pattern (workflow-{uuid}.yaml required by YamlWorkflowRepository.findAll())
  • CEL bracket notation (requires double quotes: inputs["key"] not inputs['key'])
  • Vault expression resolution (tests must call resolveVaultExpressionsInDefinition after evaluateDefinition)

Implementation vs Plan (#244)

Planned Item Status
Create InputOverrideValidationService
Fail if input key doesn't exist
Fail if type doesn't match
Typo suggestions via findClosestKey()
Update CLI path (model_method_run.ts)
Update workflow path (execution_service.ts)
Unit tests (5 planned) ✅ 16 tests

Enhancement Beyond Plan

Added definition input filtering to distinguish between:

  • Definition inputs (inputs.properties) - used with ${{ inputs.X }} expressions
  • Implicit attribute overrides - for directly setting model attributes

Without this filtering, valid definition inputs would be incorrectly rejected by inputAttributesSchema validation.

Test Plan

Manual Testing (from issue)

# Invalid key - fails with helpful message ✅
swamp model method run testEcho write --input '{"invalidKey": "value"}'
# => Unknown input key "invalidKey"

# Type mismatch - fails with type error ✅
swamp model method run testEcho write --input '{"message": 123}'
# => expected string, received number

# Valid override - succeeds ✅
swamp model method run testEcho write --input '{"message": "override"}'

Automated Testing

  • deno check - Type checking passes
  • deno lint - Linting passes
  • deno fmt - Formatting passes
  • deno run test - All 1669 tests pass
  • deno run compile - Binary compiles successfully

Implements input override validation for step/CLI inputs that override model definition attributes (implicit inputs). This
ensures users get helpful error messages when providing invalid input keys or type mismatches.

Closes #244

## Changes

### New Files
- `src/domain/inputs/input_override_validation_service.ts` - Validation service with Levenshtein-based typo detection
- `src/domain/inputs/input_override_validation_service_test.ts` - 16 unit tests covering all validation scenarios
- `integration/input_override_validation_test.ts` - Integration tests for CLI and workflow paths

### Modified Files
- `src/domain/inputs/mod.ts` - Export new service
- `src/cli/commands/model_method_run.ts` - Add validation before applying overrides
- `src/domain/workflows/execution_service.ts` - Add validation for workflow step inputs

### Integration Test Fixes
Fixed several integration tests that broke due to:
- Workflow filename pattern (`workflow-{uuid}.yaml` required by `YamlWorkflowRepository.findAll()`)
- CEL bracket notation (requires double quotes: `inputs["key"]` not `inputs['key']`)
- Vault expression resolution (tests must call `resolveVaultExpressionsInDefinition` after `evaluateDefinition`)

## Implementation vs Plan (#244)

| Planned Item | Status |
|-------------|--------|
| Create `InputOverrideValidationService` | ✅ |
| Fail if input key doesn't exist | ✅ |
| Fail if type doesn't match | ✅ |
| Typo suggestions via `findClosestKey()` | ✅ |
| Update CLI path (`model_method_run.ts`) | ✅ |
| Update workflow path (`execution_service.ts`) | ✅ |
| Unit tests (5 planned) | ✅ 16 tests |

### Enhancement Beyond Plan

Added **definition input filtering** to distinguish between:
- **Definition inputs** (`inputs.properties`) - used with `${{ inputs.X }}` expressions
- **Implicit attribute overrides** - for directly setting model attributes

Without this filtering, valid definition inputs would be incorrectly rejected by `inputAttributesSchema` validation.

## Test Plan

### Manual Testing (from issue)
```bash
# Invalid key - fails with helpful message ✅
swamp model method run testEcho write --input '{"invalidKey": "value"}'
# => Unknown input key "invalidKey"

# Type mismatch - fails with type error ✅
swamp model method run testEcho write --input '{"message": 123}'
# => expected string, received number

# Valid override - succeeds ✅
swamp model method run testEcho write --input '{"message": "override"}'
```

## Automated Testing

- deno check - Type checking passes
- deno lint - Linting passes
- deno fmt - Formatting passes
- deno run test - All 1669 tests pass
- deno run compile - Binary compiles successfully
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

This PR implements input override validation for step/CLI inputs that override model definition attributes (implicit inputs), ensuring users get helpful error messages for invalid input keys or type mismatches. The implementation is well-structured and follows the project's conventions.

✅ No Blocking Issues Found

The code is ready for merge.


Code Quality Assessment

TypeScript Strict Mode & Type Safety

  • ✅ No any types in new domain code (input_override_validation_service.ts)
  • ✅ The AnyOptions pattern in CLI commands follows the established codebase convention with proper deno-lint-ignore comments (workaround for Cliffy's type inference limitations)
  • ✅ Proper use of z.ZodTypeAny for Zod schema typing
  • ✅ Strong typing for interfaces (InputOverrideError, InputOverrideValidationResult)

Named Exports

  • ✅ All new files use named exports (no default exports)
  • ✅ Properly exported through mod.ts barrel file

Domain-Driven Design

  • InputOverrideValidationService is correctly implemented as a Domain Service - it's stateless and performs validation operations spanning multiple concerns (schema introspection, value validation, typo detection)
  • ✅ Clean separation of concerns - the service handles validation logic while CLI/workflow code handles orchestration
  • ✅ Proper error types with structured data (InputOverrideError) following the existing InputValidationError pattern

Test Coverage

  • ✅ 16 unit tests in input_override_validation_service_test.ts covering:
    • Valid inputs with correct types
    • Unknown keys
    • Typo suggestions (Levenshtein distance)
    • Type mismatches (string, number, boolean)
    • Empty inputs
    • Optional fields
    • Arrays, nested objects
    • Multiple errors
    • Defaults, records, unions, nullable fields, enums
  • ✅ Comprehensive integration tests in integration/input_override_validation_test.ts covering both CLI and workflow paths

Security

  • ✅ No security vulnerabilities identified
  • ✅ Input validation properly rejects invalid keys and types before they reach execution
  • ✅ No injection risks - uses Zod's safeParse for type validation

Suggestions (Non-Blocking)

  1. Line 262-266 in input_override_validation_service.ts: The ternary for actualType determination could be slightly more readable with a helper function, but the current implementation is clear enough.

  2. workflow_search.ts additions: The new interactive UI features (input_file_select_output.tsx, workflow_action_select_output.tsx) are well-structured but appear to be tangentially related to the main PR goal. Consider splitting UI enhancements into a separate PR for cleaner history - though this is minor since all changes work together.

  3. Experiment cleanup: The PR removes the entire experiments/ directory which appears to be a positive cleanup effort removing unused code.


Implementation Highlights

  • Typo detection: Excellent UX with Levenshtein distance-based suggestions ("mesage"Did you mean "message"?)
  • Definition input filtering: Smart distinction between definition inputs (${{ inputs.X }}) and implicit attribute overrides
  • Error messages: Comprehensive error formatting with suggestions and available keys
  • Integration test fixes: Good attention to detail fixing broken tests (workflow filename patterns, CEL bracket notation, vault expression resolution order)

The PR description is well-documented with clear implementation notes and test plan.

@stack72 stack72 merged commit 6dac2f4 into main Feb 10, 2026
5 checks passed
@stack72 stack72 deleted the feat/validate-step-input-overrides branch February 10, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate step input overrides against model schema

1 participant