Skip to content

[Bug]: blob report hash doesn't consider the --test-list cli option #39260

@akaltar

Description

@akaltar

Version

1.58.2

Steps to reproduce

In any playwright repo add a test-list.txt file with one test in it.
Run playwright --test-list="./test-list.txt" with a blob reporter
Observe the blob report's name
Chang the test-list.txt file to a different test.
run playwright --test-list="./test-list.txt" again with a blob reporter.
Observe that the blob report was overwritten/same name was used.

Expected behavior

Since --test-list can be an alternate option to --shard, it's contents should also be considered for the blob filename

Actual behavior

The blob filename doesn't change when the test-list file changes.
Observe that the blob report was overwritten/same name was used.

It's of course possible to work around this by renaming the blob file, but it would be nice if it worked out of the box

Additional context

I've filled out the env for my dev environment, but this happened in CI

Environment

System:
    OS: Windows 10 10.0.19045
    CPU: (32) x64 AMD Ryzen 9 9950X3D 16-Core Processor
    Memory: 34.83 GB / 61.61 GB
  Binaries:
    Node: 24.12.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.4 - D:\programs\yarn2\bin\yarn.CMD
    npm: 10.1.0 - C:\Program Files\nodejs\npm.CMD
    pnpm: 10.28.0 - C:\Program Files\nodejs\pnpm.CMD
    bun: 1.1.1 - C:\Users\nmaka\.bun\bin\bun.EXE
    Deno: 1.46.3 - C:\ProgramData\chocolatey\lib\deno\deno.EXE
  IDEs:
    VSCode: 1.108.2 - C:\Users\nmaka\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
    Claude Code: 2.1.2 - C:\Users\nmaka\AppData\Local\pnpm\claude.CMD
  Languages:
    Bash: 5.2.37 - C:\Program Files\Git\usr\bin\bash.EXE
  npmPackages:
    @playwright/test: catalog:testing => 1.58.1
    playwright: catalog:testing => 1.58.1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions