Skip to content

Add test run to validate no extra warnings in analyzer #2511

Closed
@sbomer

Description

@sbomer

We would like to merge the feature branch into main as soon as possible. The criteria for this are that the analyzer should not produce extra warnings compared to the linker (aside from a few specific edge cases where we expect it to). We don't need to reach full parity (tracked in #2273) to do this.

We should add a test run that runs all linker tests through the analyzer, and validates that it doesn't produce any extra warnings. For this test run, tests should not fail because of missing warnings. The linker tests should not be turned on in the regular test runs until the analyzer reaches parity with the linker for each feature area.

It might be worth using some xunit extension points for this. For example, it would be nice to write the testcases facts once and decorate them with attributes to indicate whether they are enabled only in the "no extra warnings" run, or also in the "linker parity" test run. It would also be nice to have custom testcase discovery to ensure we have full coverage (since currently, sharing a linker testcase requires writing a fact for each linker test file, making it easy to miss tests).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions