Improve unknown command-line option guidance

[Amaury Levé (Evangelink)]

Summary

• suggest a uniquely matching command-line option for likely typos
• identify the extension package that auto-registers known missing options
• cover first-party dump, report, retry, and video extensions plus Microsoft code coverage
• avoid suggestions for hidden options, ambiguous matches, tool-inapplicable options, and options requiring explicit framework registration

Validation

• restored the repository toolchain and dependencies after merging main
• passed all 69 CommandLineHandlerTests on net8.0

[github-actions]

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

✅ 22/22 dimensions clean — no findings.

Summary: This PR adds "did you mean" suggestions and known-extension package hints for unknown CLI options. The implementation is well-scoped:

• KnownExtensionOptionsByPackage is a static readonly Dictionary initialized once — safe for concurrent reads.
• The Damerau-Levenshtein edit distance and LINQ enumeration are only invoked on the error path (unknown options), so the allocations are acceptable.
• All new code is private static / internal — no public API surface changes.
• Resource strings are properly added to .resx with correct {0}/{1} placeholders and no {Locked} concerns; .xlf files are regenerated.
• Tests cover: close-match suggestion, ambiguous-match suppression, exact known-extension match, misspelled extension option (both with and without the extension registered), and all 11 extension packages.

[Copilot]

Pull request overview

Improves unknown CLI option errors with typo suggestions and missing-extension package guidance.

Changes:

• Adds edit-distance matching while excluding hidden and ambiguous candidates.
• Maps known extension options to their NuGet packages.
• Adds localized resources and unit tests.

Show a summary per file

File	Description
CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs	Implements option and package suggestions.
PlatformResources.resx	Adds guidance messages.
PlatformResources.cs.xlf	Updates Czech localization resources.
PlatformResources.de.xlf	Updates German localization resources.
PlatformResources.es.xlf	Updates Spanish localization resources.
PlatformResources.fr.xlf	Updates French localization resources.
PlatformResources.it.xlf	Updates Italian localization resources.
PlatformResources.ja.xlf	Updates Japanese localization resources.
PlatformResources.ko.xlf	Updates Korean localization resources.
PlatformResources.pl.xlf	Updates Polish localization resources.
PlatformResources.pt-BR.xlf	Updates Portuguese localization resources.
PlatformResources.ru.xlf	Updates Russian localization resources.
PlatformResources.tr.xlf	Updates Turkish localization resources.
PlatformResources.zh-Hans.xlf	Updates Simplified Chinese resources.
PlatformResources.zh-Hant.xlf	Updates Traditional Chinese resources.
CommandLineHandlerTests.cs	Tests typo and extension guidance behavior.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs:193

• When a tool is selected, ValidateAsync first filters out every non-tool provider (CommandLineOptionsValidator.cs:55-59). Therefore a command such as tool --report-trx reaches this branch and says to add Microsoft.Testing.Extensions.TrxReport, even if that package/provider is already registered—and adding it cannot make a normal extension option valid in tool mode. Please suppress package-reference guidance for tool invocations, or retain the unfiltered provider set so this path can distinguish a missing package from an option that is unavailable in the selected command scope.

        string? packageName = GetKnownExtensionPackage(unknownOptionName);
        if (packageName is not null)
        {
            AppendMissingExtensionSuggestion(stringBuilder, unknownOptionName, packageName);

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[github-actions]

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 100.6 AIC · ⌖ 1.14 AIC · ⊞ 16.9K · ◷

[Copilot]

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs:104

• The <= 12 => 2 and _ => 3 thresholds are not exercised by the added tests: every positive case is only one edit away, so changing both branches to return 1 would leave all 50 tests passing. Add cases where a 5–12 character option requires exactly two edits and a >12 character option requires exactly three edits (plus just-over-threshold negatives) so these user-visible boundaries cannot regress.

        ],
        ["Microsoft.Testing.Extensions.VideoRecorder"] =
        [
            "capture-video",
            "capture-video-args",

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (1)

src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs:204

• This adds every catalogued extension option to the suggestion candidates even when its provider is not registered, while the PR description says suggestions only consider visible options registered for the current invocation and avoid unregistered options. The new package-guidance tests confirm that the implementation intentionally does the opposite. Please update either the implementation or the PR description so the documented behavior matches what ships.

        string? suggestedOptionName = FindSuggestedOption(
            unknownOptionName,
            visibleOptionNames.Concat(KnownExtensionOptionsByPackage.Values.SelectMany(static optionNames => optionNames)));

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[github-actions]

🧵 Parallel-safety audit — PR #10798

Parallelization — one row per test assembly audited:

Test assembly	Scope	Workers	Analyzer coverage
Microsoft.Testing.Platform.UnitTests	MethodLevel ([assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in Program.cs, unchanged by this PR)	CPU count	coverable once the parallel-safety analyzers ship (attribute-based opt-in)

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — no severities to report.

The only changed test file is CommandLineHandlerTests.cs, which adds ~15 new [TestMethod]s to CommandLineHandlerTests plus a constructor-arg tweak on the private ExtensionCommandLineProviderMockValidConfiguration mock (added isHidden parameter). All new/changed test bodies:

• Only construct local values (CommandLineParseResult, arrays of ICommandLineOptionsProvider, a SystemEnvironment instance) and call CommandLineOptionsValidator.ValidateAsync(...), a pure function over its arguments.
• Read only instance fields (_systemCommandLineOptionsProviders, _extensionCommandLineOptionsProviders), which are freshly constructed per test instance (MSTest constructs the test class per test) — not shared/static state.
• Perform no filesystem I/O, no environment-variable or CWD mutation, no console redirection, no culture mutation, and touch no static fields.
• Declare no [ResourceLock] / [DoNotParallelize], and none is needed — nothing here is process-global.

Nothing in this diff introduces a parallel-safety hazard, and the pre-existing test class (already at MethodLevel via the unchanged assembly attribute) shows no cross-test shared-resource pattern to flag either.

_{Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.}

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 39.2 AIC · ⌖ 3.08 AIC · ⊞ 24.8K · [◷]( · ◷)

[github-actions]

🧪 Expert test review — PR #10798

Grade	Test	Mutation	Notes	How to improve
B (80–89)	new CommandLineHandlerTests. ParseAndValidateAsync_ KnownExtensionOptionWithoutExtension_ SuggestsPackage	2/2 killed	Verifies unknown+package-hint text but never asserts a "Did you mean" line is absent for these exact matches.	Add Assert.DoesNotContain("Did you mean", result.ErrorMessage) to lock in the exact-match path.
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ KnownExtensionOptionInToolMode_ DoesNotSuggestPackageOrOption	3/3 killed	Correctly proves tool-mode filters out both catalog package hints and typo suggestions for an inapplicable non-tool provider.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ UnknownOptionWithCloseRegisteredOption_ SuggestsOption	2/2 killed	Full-message AreEqual pins the exact unknown/suggestion/hint/command-line text.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ UnknownOption_ RespectsSuggestionDistanceThresholds	4/4 killed	Data-driven boundary cases exercise both sides of the length-scaled edit-distance thresholds.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ UnknownOptionWithAmbiguousMatches_ DoesNotSuggestOption	1/1 killed	Confirms two equidistant candidates correctly suppress a misleading single suggestion.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ UnknownOptionWithCloseHiddenOption_ DoesNotSuggestOption	1/1 killed	Targets the hidden-option filter directly, addressing the earlier reviewer regression request.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ MisspelledKnownExtensionOption_ SuggestsOptionAndPackage	3/3 killed	Data-driven cases verify both the corrected option name and its owning package appear together.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ MisspelledRegisteredExtensionOption_ DoesNotSuggestPackage	2/2 killed	Full-message assertion confirms a registered (non-catalog) provider never gets a package hint.	—
A (90–100)	new CommandLineHandlerTests. ParseAndValidateAsync_ MisspelledJsonOption_ SuggestsRegisteredOption	1/1 killed	Confirms the testconfig.json path reuses the same unknown/suggestion diagnostic, prefixed correctly.	—

Summary: The nine new/modified tests in CommandLineHandlerTests.cs cover the new "did you mean" suggestion and known-extension-package guidance logic in CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs thoroughly: exact-distance thresholds, ambiguous ties, hidden-option exclusion, tool-mode suppression, and the testconfig.json path are all exercised with focused Arrange-Act-Assert bodies and meaningful assertions (mostly full-message Assert.AreEqual, some scoped Assert.Contains/DoesNotContain). No correctness, reliability, or convention issues were found. Only one test (KnownExtensionOptionWithoutExtension_SuggestsPackage) could add a negative assertion for extra rigor, though the current production code path makes that gap non-exploitable today.

_{This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Suggestions on the Files
changed tab can be applied with one click. Re-run with
/review-tests.}

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 118.8 AIC · ⌖ 0.987 AIC · ⊞ 16.9K · [◷]( · ◷)

[Copilot]

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Balanced