test: add workflow spec blackbox coverage

[yohamta0]

Summary

Adds workflow specs and blackbox validation coverage for the current workflow behavior.

Changes

• Add numbered workflow specs for project layout, YAML schema, value resolution, field evaluation, dynamic evaluation, step references, outputs, and run behavior
• Add validate command prevalidation for YAML stream shape and root fields
• Add blackbox test harness, schema fixtures, and GitHub Actions workflow for binary-level validation checks
• Address review feedback for checkout credential persistence, resolved validate paths, spec wording, and test assertions

Related Issues

N/A

Testing

• make bin
• make lint
• go test ./internal/cmd -count=1
• go test ./tests -count=1
• DAGU_BIN=.local/bin/dagu go test -count=1 ./tests

Checklist

• Code follows the project style guidelines
• Self-review of the code has been performed
• Tests have been added or updated as needed
• Documentation has been updated as needed
• Changes have been tested locally

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds eight numbered data-plane specification documents, an implementation guide, and a spec README. Implements AST-based YAML structure validation in the validate CLI command. Introduces a black-box test runner with isolated environment support, schema conformance tests with YAML fixtures, and a CI workflow to execute them. Also fixes mutex handling in two unrelated service monitor tests.

Changes

Spec corpus, validate implementation, and black-box tests

Layer / File(s)	Summary
Spec corpus: README, implementation guide, and numbered specs 001–008 specs/README.md, specs/IMPLEMENTATION.md, specs/001-project.md, specs/002-yaml-schema.md, specs/003-value-resolution.md, specs/004-step-reference.md, specs/005-field-evaluation.md, specs/006-dynamic-evaluation.md, specs/007-step-outputs.md, specs/008-step-run.md	Introduces meta-documents (README, IMPLEMENTATION guide) and eight numbered spec files defining project structure, YAML schema, value resolution, step identity, field evaluation, dynamic evaluation, step outputs, and step run semantics including acceptance criteria for black-box fixtures.
validate CLI: AST-based YAML structure validation internal/cmd/validate.go	Adds os and go-yaml AST imports; updates the Cobra command with SilenceUsage; wires a pre-pass call to validateWorkflowFile before DAG loading; implements all internal helpers for document root checking, entrypoint/name/uniqueness enforcement, non-empty steps sequence validation, and aggregated error formatting.
Black-box test runner infrastructure tests/runner_test.go	Adds Runner and Result types with a constructor that copies testdata into a temp directory, a Run method that executes DAGU_BIN with a 10s timeout capturing stdout/stderr/exit code, assertion methods, binary resolution logic, and an isolatedEnv helper that sanitizes the process environment.
Schema conformance test and YAML fixtures tests/002_schema_test.go, tests/testdata/002_schema/*	Adds Test002Schema exercising valid and invalid YAML fixtures via the runner, asserting exit codes, empty stdout, stderr substrings, absence of the Usage: line, and non-execution of steps. Adds all corresponding testdata YAML files covering minimal valid, inline sub-DAG, and each invalid scenario.
CI: black-box test workflow .github/workflows/blackbox-tests.yml	Adds the blackbox-tests.yml GitHub Actions workflow triggered on push to main and pull request events with path filters; pins Go to 1.26.3; builds the binary via make bin; runs go test -count=1 ./tests with DAGU_BIN set.

Service monitor test fixes

Layer / File(s)	Summary
chatbridge: assert readCalls increases after destination added internal/service/chatbridge/monitor_external_test.go	Snapshots readCalls before setDestinations, then uses require.Eventually to assert readCalls increases before proceeding to delivery assertions.
telegram: explicit mutex lock in Eventually closure internal/service/telegram/monitor_test.go	Refactors the require.Eventually closure to lock service.mu explicitly, compute separate attempt counts per session, unlock, then return the combined condition.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• dagucloud/dagu#2122: Both PRs modify internal/cmd/validate.go to add validate-time logic — this PR adds AST-based YAML structure pre-validation while the retrieved PR adds deprecation warning collection after DAG load.
• dagucloud/dagu#2272: The Test002Schema black-box fixtures and the validate implementation in this PR are directly tied to the specs and black-box test foundation introduced in that PR.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 4.76% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'test: add workflow spec blackbox coverage' clearly summarizes the main change—adding blackbox tests for workflow specs—and accurately reflects the primary focus of the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The pull request description follows the template structure, includes all required sections (Summary, Changes, Related Issues, Testing, and Checklist), and provides clear context about the additions of workflow specs and blackbox validation coverage.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch blackbox-tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/blackbox-tests.yml:
- Around line 46-47: The "Check out code" step using actions/checkout is missing
the persist-credentials configuration, which could leave the workflow token in
git config for subsequent steps. Add the persist-credentials: false option to
the checkout action to explicitly disable credential persistence and improve
security by ensuring the workflow token is not retained after the checkout step
completes.

In `@internal/cmd/validate.go`:
- Around line 80-83: In the validation block where validateWorkflowFile is
called with dag.Location, the formatValidationErrors call on the error return
statement is using args[0] instead of dag.Location. Replace args[0] with
dag.Location in the formatValidationErrors function call to ensure the error
message displays the correct file path that was actually validated rather than
the command-line argument which may differ for discovered project targets.

In `@specs/004-step-reference.md`:
- Line 30: The regex pattern `[a-z][a-z0-9_]*` for the step id validation does
not properly enforce snake_case rules because it permits consecutive underscores
and trailing underscores. Replace this regex with a pattern that enforces true
snake_case by requiring lowercase letters or digits after each underscore and
preventing multiple consecutive underscores or trailing underscores at the end
of the identifier.

In `@specs/005-field-evaluation.md`:
- Line 141: The acceptance criterion at line 141 contradicts the behavior
section by stating that $() inside params[].eval should be preserved rather than
executed. Since the spec identifies params[].eval as the command-substitution
surface, $() must be executed there, not left untouched. Update line 141 to
specify that a black-box fixture verifies $() in params[].eval is executed by
Dagu, aligning the acceptance criterion with both the behavior description and
the provided examples.

In `@tests/runner_test.go`:
- Around line 95-143: The five assertion helper methods (ExpectExitCode,
ExpectStdout, ExpectStderr, ExpectStderrContains, and ExpectStderrNotContains)
in the Result type are using manual t.Fatalf checks instead of stretchr/testify
assertions. Replace the manual comparison in ExpectExitCode with require.Equal
for the exit code, use require.Equal in ExpectStdout for stdout comparison, use
require.Equal in ExpectStderr for stderr comparison, use require.Contains in
ExpectStderrContains to check for substring presence in the loop, and use
require.NotContains in ExpectStderrNotContains to check for substring absence in
the loop. Make sure to import the stretchr/testify/require package at the top of
the file if not already present.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 50de00c6-5bad-438f-a404-ff517ef73eaf

📥 Commits

Reviewing files that changed from the base of the PR and between 942e4d4 and cda4428.

📒 Files selected for processing (27)

• .github/workflows/blackbox-tests.yml
• internal/cmd/validate.go
• internal/service/chatbridge/monitor_external_test.go
• internal/service/telegram/monitor_test.go
• specs/001-project.md
• specs/002-yaml-schema.md
• specs/003-value-resolution.md
• specs/004-step-reference.md
• specs/005-field-evaluation.md
• specs/006-dynamic-evaluation.md
• specs/007-step-outputs.md
• specs/008-step-run.md
• specs/IMPLEMENTATION.md
• specs/README.md
• tests/002_schema_test.go
• tests/runner_test.go
• tests/testdata/002_schema/duplicate_document_names.yaml
• tests/testdata/002_schema/duplicate_root_keys.yaml
• tests/testdata/002_schema/empty_document.yaml
• tests/testdata/002_schema/empty_steps.yaml
• tests/testdata/002_schema/entrypoint_name_forbidden.yaml
• tests/testdata/002_schema/inline_subdag_valid.yaml
• tests/testdata/002_schema/later_document_without_name.yaml
• tests/testdata/002_schema/minimal_valid.yaml
• tests/testdata/002_schema/missing_steps.yaml
• tests/testdata/002_schema/steps_mapping_forbidden.yaml
• tests/testdata/002_schema/unknown_root_field.yaml

[cubic-dev-ai]

1 issue found and verified against the latest diff

<sub>Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic</sub>