Testing Tracking Issue

[cwfitzgerald]

This is all about our test suite and how to improve it.

Missing Tests

General

• Add copy_texture_to_texture tests #3315
• Test Resource Conflict Validation #3132
• Test Push Constant Validation #3133
• Test Notable Special Shader Functions #3159
• Test Buffer to Buffer and Buffer to/from Image Copies with Problematic Usages #3161
• Tests for Scissors #3871
• Port Extraction Tests to Tests #4239
• Missing test coverage for draw_indirect #4256
• Add unit tests for compressed texture formats #7666

Features

These aren't part of WebGPU, or part of firefox's WebGPU impl currently, so should have more aggressive testing.

• PIPELINE_STATISTICS_QUERY
• Bindless
  • BindGroupLayoutEntry::count Validation Does Not Check That Shader Uses Binding Arrays #3648
  • Array binding provided zero elements validation error when using PARTIALLY_BOUND_BINDING_ARRAY #3692
  • Validation for shader types
  • Execution tests for all kinds of bindless resources
• Multi-draw Indirect
• Push constants
• Address modes
• Polygon modes
• Tests for multiview support #7196
• Shader primitive index
• Early depth test
• Dual source blending

Infrastructure

• Use wasm-compatible asserts in tests #3286
• Consider Testing Against Mobile Devices using BrowserStack #3836
• WASM Tests Unsupported on WebGPU #3677
• Hard to Tell if Test is Skipped #3680
• WASM Image Tests Do Nothing #4290
• Enable DX12 GPU Based Validation #4291
• Expose Ability to Turn Important Private Capabilities Off #4582

CI

• Reconsider lavapipe #3622
• Linux CPU Implementations Crashes Sporatically on all Tests #3250

Failing Tests

• Early frees on CPU Implementations #3193
• AMD on DX12 Renders Nothing on msaa-line Example #3838
• Incorrect indices for push constants in webgl/gles backend #3910 (This is ready to go, just needs to be MR'd)
• Dropping a failed QuerySet within an error scope panics #4139
• ComputePipeline::get_bind_group_layout panics if the pipeline's module is invalid #4167
• draw_instanced_offset fails on GL #4276
• Multithreaded Compute Segfaults in CI on GL/Linux #4285
• Matrix of the form matCx2 not getting translated properly when in a uniform buffer #4371
• With Restricted Image Loads, Invalid Spirv Generated #4581
• Make SAMPLER_CREATION_FAILURE test case pass on Windows WARP #7805