Comparison images generated while running the task to record screenshots with the support of ComposePreviewScanner to scan the Composable Previews

[cescko]

I have found that in some cases *_actual.png and *_compare.png comparison screenshots are generated while running the task to record the screenshots in the build/outputs/roborazzi folder.

I managed to replicate the case in a sample project and described the steps in the README file. It should be easily replicable.

It seems that the problem could be depend on to the test generated via the generateComposePreviewRobolectricTests configuration in build.gradle.kts file. I had no problems generating the screenshots of the previews with a written test without using the configuration task.

[takahirom]

Thank you for preparing the reproducing project.
I tried without generateComposePreviewRobolectricTests, and it seems there is the same problem.
Record -> Change preview -> Verify -> Remove the actual and compare PNGs -> Record -> Actual and compare PNGs exists.

There is a directory build/intermediate/roborazzi, which is used for caching. The test task's output is in build/intermediate/roborazzi, while build/output/roborazzi serves as the input directory for the test task. Therefore, we need to copy images to build/intermediate/roborazzi to use caches when finishing tests. The reason we copy to build/output/roborazzi is also related to caching, but I forgot the exact details. We have many tests for the cache, so I think we can find more information in #564.

Anyway, I suggest you try setting roborazzi.cleanupOldScreenshots=true. Could you check it out?
https://github.com/takahirom/roborazzi?tab=readme-ov-file#roborazzicleanupoldscreenshots

[takahirom]

Now I understand that we copy to build/output/roborazzi. When we use the cache for the test task, images in build/*intermediate*/roborazzi are restored by Gradle, but build/*output*/roborazzi will be empty because it is not an output directory. That's why we copy images to build/output/roborazzi.

[cescko]

@takahirom Thanks, we are evaluating your proposal and also checking again the Gradle tasks behaviour. We will get back to you as soon as possible! Thanks!