Repository navigation
Fix test_all_artifacts_have_required_fields silently skipping half th… - #1
Merged
scriptedstatement merged 2 commits intoOct 1, 2026
Merged
Conversation
…e artifacts
The test derived a filename from each artifact's name field and skipped the
artifact when the lookup missed:
art = loader.get_artifact(name.lower().replace(" ", "_").replace("-", "_"))
if art is None:
continue
That only resolves when an artifact's name round-trips to its filename, which
is not a property the data has or should need. 'Authentication Log' derives
authentication_log but the file is auth_log.yaml; 'Cron Jobs' derives cron_jobs
against crontab.yaml; 'Login Records' derives login_records against
wtmp_utmp.yaml. Each miss hit the continue and the artifact went unchecked
without any signal.
The result was that a test named test_all_artifacts_have_required_fields
validated 27 of 53 artifacts — 22 Windows and 4 Linux were skipped.
Read the artifact directories directly instead, matching the seven other
_load_all_in_dir loops already in this file. Coverage goes to all 53 and the
test passes unchanged, since every artifact does carry name, description and
platform. This is the only artifact test in the file that was not already
using that idiom.
The platform tuple includes "macos" to match loader.py, which enumerates
("windows", "linux", "macos") in both list_artifacts and get_artifacts_for_tool.
_load_all_in_dir returns an empty list for a directory that does not exist, so
this is inert today and the test picks up macOS artifacts automatically if that
directory is ever added — rather than silently skipping them, which is the bug
this commit fixes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mpilking
force-pushed
the
fix-artifact-schema-test
branch
from
September 9, 2026 03:11
86b7f3c to
3fb69d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test_all_artifacts_have_required_fieldsvalidates 27 of 53 artifacts. The other 26 are silently skipped.The test derives a filename from each artifact's
namefield, then gives up when the lookup misses:That only resolves when an artifact's name directly derives to its filename, which often fails:
Every miss hits the
continueand the artifact goes unchecked with no signal. 22 Windows and 4 Linux artifacts are currently affected.Fix
Read the artifact directories directly, matching the seven other
_load_all_in_dir loopsalready in this file — this was the only artifact test not using that idiom.Result
Coverage goes from 27 to all 53 artifacts, and the test passes unchanged: every artifact does carry name, description and platform. No artifact files are modified.
🤖 Generated with Claude Code (https://claude.com/claude-code)