A library to help auto-generate screenshot tests from Composable Previews with any screenshot testing library: JVM-based (i.e. Paparazzi, Roborazzi) as well as Instrumentation-based (i.e. Shot, Dropshots, Android-Testify, etc.)
Composable Preview Scanner | Showkase | Compose Preview Screenshot Testing | |
---|---|---|---|
Independent of AGP version | β | β | β |
Library-agnostic solution | β | β | β1 |
Scans previews in "main" source | β | β | β2 |
Preview Infos available | β | β3 | β |
Specific Config (e.g. for Libs) available | β 4 | β | |
Compose Multiplatform Previews support | *β 6 | β7 | β |
1 Compose Preview Screenshot Testing is a standalone solution based on LayoutLib, whereas ComposablePreviewScanner and Showkase provide Composables' infos so you can run screenshot tests with your favourite screenshot testing library.
2 Compose Preview Screenshot Testing requires to put the previews in a brand-new "screenshotTest" source. ComposablePreviewScanner and Showkase only work with previews in the "main" source, so if you want to have the previews used for screenshot tests separate, I recommend to create an extra module for them and their screenshot tests and do not include that module in :app.
3 Showkase components only hold information about the Composable, but not about the Preview Info (i.e. ApiLevel, Locale, UiMode, FontScale...).
4 ComposablePreviewScanner supports adding extra lib-config (e.g. Paparazzi's Rendering Mode or Roborazzi's compare options) in the form of annotations that are additionally added to the preview. You can check how in the examples below in Jvm Screenshot Tests and Instrumentation Screenshot Tests respectively.
5 Compose Preview Screenshot Testing supports only general tolerance via gradle plugin from version 0.0.1-alpha06
6 ComposablePreviewScanner provides some Compose Multiplatform support: It can scan @Preview
s in common
, as described in the Compose-Multiplatform section. Compose-Desktop @Preview
s are still not supported out of the box because of this issue, but you can check the Compose-Desktop support section for a workaround.
7 Showkase: Compose Multiplatform Support
ComposablePreviewScanner also works with:
@PreviewParameters
- Multi-Previews, including
@PreviewScreenSizes
,@PreviewFontScales
,@PreviewLightDark
, and@PreviewDynamicColors
. - private
@Previews
(from version 0.1.3+) @Previews
inside public classes1 (from version 0.3.0+), not nested classes though
but does not work with
@Previews
that are not located in the "main" source
1 Compose Preview Screenshot Testing tool requires to put your @Previews
inside a class. By supporting this, you can keep @Previews
in a separate class, and keep that class in the "main" source to auto-generate screenshot tests from them with ComposablePreviewScanner and Roborazzi, Paparazzi or any instrumentation-based screenshot testing library, or move that class to "screenshotTest" source to auto-generate screenshot tests from them with Compose Preview Screenshot Testing.
Warning
Beware the prefixes:
Maven Central -> io.github
Jitpack -> com.github
dependencies {
// jvm tests (e.g. with Roborazzi & Paparazzi)
testImplementation("io.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// instrumentation tests (e.g. with Shot, Dropshots & Android-Testify)
debugImplementation("io.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// compose multiplatform (jvm-targets)
testImplementation("io.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
}
Add jitpack to your root build.gradle file:
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
dependencies {
// jvm tests (e.g. with Roborazzi & Paparazzi)
testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// instrumentation tests (e.g. with Shot, Dropshots & Android-Testify)
debugImplementation("com.github.sergio-sastre.ComposablePreviewScanner:android:<version>")
// compose multiplatform (jvm-targets), from 0.2.0
testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
}
- Jvm Screenshot Tests
1.1 Paparazzi
1.2 Roborazzi - Instrumentation Screenshot Tests
- Compose Multiplatform Support
The API is pretty simple:
AndroidComposablePreviewScanner() // or CommonComposablePreviewScanner(), see Compose Multiplatform section
.scanPackageTrees(
include = listOf("your.package", "your.package2"),
exclude = listOf("your.package.subpackage1", "your.package2.subpackage1")
)
// options to filter scanned previews
.excludeIfAnnotatedWithAnyOf(
ExcludeForScreenshot::class.java,
ExcludeForScreenshot2::class.java
)
.includeAnnotationInfoForAllOf(
ScreenshotConfig::class.java,
ScreenshotConfig2::class.java
)
.includePrivatePreviews() // Otherwise they are ignored
.filterPreviews {
// filter by any previewInfo: name, group, apiLevel, locale, uiMode, fontScale...
previewInfo -> previewInfo.apiLevel == 30
}
// ---
.getPreviews()
There are 2 more options to scan previews:
- All Packages:
scanAllPackages()
. This might require a huge amount of memory since it would scan not only in a set of packages, but in all packages used in your app/module (i.e. also in its transitive dependencies). This is in 99% of the cases unnecessary, and scanning the main package trees of your module should be sufficient. - From a file containing the ScanResult. This speeds up your screenshot tests, since avoids the time-consuming process of scanning each time by reusing the previously scanned data:
2.1.scanFile(jsonFile: File)
. Use this for JVM-based screenshot testing libraries (i.e. Roborazzi & Paparazzi).
2.2.scanFile(inputStream: InputStream)
. This is useful for Instrumentation-based screenshot testing libraries.
To dump the ScanResult into a File, the easiest is to create a unit test for that. Likely to be provided via gradle task in the future:
class SaveScanResultInFiles {
@Test
fun `task -- save scan result in file`() {
val scanResultFileName = "scan_result.json"
ScanResultDumper()
.scanPackageTrees("my.package")
// for unit tests
.dumpScanResultToFile(scanResultFileName)
// for instrumentation tests
.dumpScanResultToFileInAssets(
flavourName = "myFlavour",
fileName = scanResultFileName
)
}
}
Warning
You'll have to ensure that the file is already generated before running the screenshot tests. You'll also need to keep it up to date.
You can find executable examples here
Let's say we want to enable some custom Paparazzi Config for some Previews, for instance a maxPercentDifferent value
- Define your own annotation for the Lib config.
annotation class PaparazziConfig(val maxPercentDifferent: Double)
- Annotate the corresponding Previews accordingly (you do not need to annotate all):
@PaparazziConfig(maxPercentDifferent = 0.1F)
@Preview(uiMode = Configuration.UI_MODE_NIGHT_YES)
@Composable
fun MyComposable(){
...
}
- Include your annotation info in the Preview
object ComposablePreviewProvider : TestParameter.TestParameterValuesProvider {
override fun provideValues(): List<ComposablePreview<AndroidPreviewInfo>> =
AndroidComposablePreviewScanner()
.scanPackageTrees("my.package", "my.package2")
.includeAnnotationInfoForAllOf(PaparazziConfig::class.java)
... // any other filtering option
.getPreviews()
}
- Map the PreviewInfo and PaparazziConfig values. For instance, you can use a custom class for that.
// The DevicePreviewInfoParser used in this method is available since ComposablePreviewScanner 0.4.0
object DeviceConfigBuilder {
fun build(previewDevice: String): DeviceConfig {
val parsedDevice = DevicePreviewInfoParser.parse(device)?.inPx() ?: return DeviceConfig()
return DeviceConfig(
screenHeight = parsedDevice.dimensions.height.toInt(),
screenWidth = parsedDevice.dimensions.width.toInt(),
density = Density(parsedDevice.densityDpi),
xdpi = parsedDevice.densityDpi, // not 100% precise
ydpi = parsedDevice.densityDpi, // not 100% precise
size = ScreenSize.valueOf(parsedDevice.screenSize.name),
ratio = ScreenRatio.valueOf(parsedDevice.screenRatio.name),
screenRound = ScreenRound.valueOf(parsedDevice.shape.name),
orientation = when (parsedDevice.orientation) {
Orientation.PORTRAIT -> ScreenOrientation.PORTRAIT
Orientation.LANDSCAPE -> ScreenOrientation.LANDSCAPE
}
)
}
}
object PaparazziPreviewRule {
fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): Paparazzi =
Paparazzi(
deviceConfig = DeviceConfigBuilder.build(preview.previewInfo.device).copy(
nightMode =
when(preview.previewInfo.uiMode and Configuration.UI_MODE_NIGHT_MASK == Configuration.UI_MODE_NIGHT_YES) {
true -> NightMode.NIGHT
false -> NightMode.NOTNIGHT
}
... // other configurations
),
maxPercentDifference = preview.getAnnotation<PaparazziConfig>()?.maxPercentDifference ?: 0F
)
}
- Create the corresponding Parameterized Test:
@RunWith(TestParameterInjector::class)
class PreviewTestParameterTests(
@TestParameter(valuesProvider = ComposablePreviewProvider::class)
val preview: ComposablePreview<AndroidPreviewInfo>,
) {
@get:Rule
val paparazzi: Paparazzi = PaparazziPreviewRule.createFor(preview)
@Test
fun snapshot() {
// Recommended for more meaningful screenshot file names. See #Advanced Usage
val screenshotId = AndroidPreviewScreenshotIdBuilder(preview).ignoreClassName().build()
paparazzi.snapshot(name = screenshotId) {
preview()
}
}
}
- Run these Paparazzi tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordPaparazziDebug
You can find executable examples here
Let's say we want to enable some custom Roborazzi Config for some Previews, for instance a maxPercentDifferent value
- Define your own annotation for the Lib Config.
annotation class RoborazziConfig(val comparisonThreshold: Double)
- Annotate the corresponding Previews accordingly:
@RoborazziConfig(comparisonThreshold = 0.1)
@Preview(uiMode = Configuration.UI_MODE_NIGHT_YES)
@Composable
fun MyComposable(){
...
}
- Map the PreviewInfo and RoborazziConfig values. For instance, you can use a custom class for that.
object RoborazziOptionsMapper {
fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): RoborazziOptions =
preview.getAnnotation<RoborazziConfig>()?.let { config ->
RoborazziOptions(
compareOptions = CompareOptions(resultValidator = ThresholdValidator(config.comparisonThreshold))
)
} ?: RoborazziOptions()
}
object RobolectricPreviewInfosApplier {
fun applyFor(preview: ComposablePreview<AndroidPreviewInfo>) {
// Set the device from the preview info. Available since ComposablePreviewScanner 0.4.0
RobolectricDeviceQualifierBuilder.build(preview.previewInfo)?.run {
RuntimeEnvironment.setQualifiers(this)
}
val uiMode = nightMode =
when(preview.previewInfo.uiMode and Configuration.UI_MODE_NIGHT_MASK == Configuration.UI_MODE_NIGHT_YES) {
true -> "+night"
false -> "+notnight"
}
RuntimeEnvironment.setQualifiers(uiMode)
... // other configurations
}
}
Check the following link for a full list of Robolectric device qualifiers and this blog post on how to set the cumulative Qualifiers dynamically
- Create the corresponding Parameterized Test:
@RunWith(ParameterizedRobolectricTestRunner::class)
class PreviewParameterizedTests(
private val preview: ComposablePreview<AndroidPreviewInfo>,
) {
companion object {
// Optimization: This avoids scanning for every test
private val cachedPreviews: List<ComposablePreview<AndroidPreviewInfo>> by lazy {
AndroidComposablePreviewScanner()
.scanPackageTrees("your.package", "your.package2")
.filterPreviews { previewParams -> previewParams.apiLevel == 30 }
.includeAnnotationInfoForAllOf(RoborazziConfig::class.java)
.getPreviews()
}
@JvmStatic
@ParameterizedRobolectricTestRunner.Parameters
fun values(): List<ComposablePreview<AndroidPreviewInfo>> = cachedPreviews
}
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [30]) // same as the previews we've filtered
@Test
fun snapshot() {
RobolectricPreviewInfosApplier.applyFor(preview)
// Recommended for more meaningful screenshot file names. See #Advanced Usage
val screenshotId = AndroidPreviewScreenshotIdBuilder(preview).build()
captureRoboImage(
filePath = "${screenshotId}.png",
roborazziOptions = RoborazziOptionsMapper.createFor(preview)
) {
preview()
}
}
}
- Run these Roborazzi tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziDebug
You can find executable examples that use ComposablePreviewScanner with the different instrumentation-based libraries in the corresponding links below:
Android does not use the standard Java bytecode format and does not actually even have a runtime classpath. Therefore, the simplest way to support instrumentation tests, is to...
- run the scan in a unit test & save it in a file accessible by instrumentation tests e.g. in assets
class SaveScanResultInAssets {
@Test
fun `task -- save scan result in assets`() {
val scanResultFileName = "scan_result.json"
ScanResultDumper()
.scanPackageTrees("my.package")
.dumpScanResultToFileInAssets(
fileName = scanResultFileName
)
assert(
assetsFilePath(
fileName = scanResultFileName
).exists())
}
}
Ensure that the .json with the scan result is up-to-date before executing the instrumentation screenshot tests. For instance, execute that test always before your instrumentation screenshot tests. Ideally, this scanning could be done via a Gradle Plugin in the future instead of by running it in a unit test.
- Now proceed to prepare your Composable Preview Tests with, for instance, Dropshots.
Let's say we want to enable some custom Dropshots Config for some Previews, for instance a maxPercentDifferent value.
- Define your own annotation
annotation class DropshotsConfig(val comparisonThreshold: Double)
- Annotate the corresponding Previews accordingly:
@DropshotsConfig(comparisonThreshold = 0.15) @Preview(uiMode = Configuration.UI_MODE_NIGHT_YES) @Composable fun MyComposable(){ ... }
- Map the PreviewInfo and DropshotsConfig values. For instance, you can use a custom class for that. To map the Preview Info values, I recommend to use the ActivityScenarioForComposableRule of AndroidUiTestingUtils
object DropshotsPreviewRule { fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): Dropshots = preview.getAnnotation<DropshotsConfig>()?.let { config -> Dropshots( resultValidator = ThresholdValidator(config.comparisonThreshold)) ) } ?: Dropshots() } object ActivityScenarioForComposablePreviewRule { fun createFor(preview: ComposablePreview<AndroidPreviewInfo>): ActivityScenarioForComposableRule { val uiMode = when(preview.previewInfo.uiMode and Configuration.UI_MODE_NIGHT_MASK == Configuration.UI_MODE_NIGHT_YES) { true -> UiMode.NIGHT false -> UiMode.DAY } return ActivityScenarioForComposableRule( config = ComposableConfigItem( uiMode = uiMode ... // other configurations ) ) } }
- Create the corresponding Parameterized Test:
@RunWith(ParameterizedTestRunner::class) class PreviewParameterizedTests( private val preview: ComposablePreview<AndroidPreviewInfo>, ) { companion object { private val cachedPreviews: List<ComposablePreview<AndroidPreviewInfo>> by lazy { AndroidComposablePreviewScanner() .scanFile(getInstrumentation().context.assets.open("scan_result.json")) .includeAnnotationInfoForAllOf(DropshotsConfig::class.java) .getPreviews() } @JvmStatic @ParameterizedTestRunner.Parameters fun values(): List<AndroidComposablePreview<PreviewInfo>> = cachedPreviews } @get:Rule val dropshots: Dropshots = DropshotsPreviewRule.createFor(preview) @get:Rule val activityScenarioForComposable: ActivityScenarioForComposableTestRule = ActivityScenarioForComposablePreviewTestRule.createFor(preview) @Test fun snapshot() { activityScenarioForComposableRule.onActivity { it.setContent { preview() } } dropshots.assertSnapshot( view = activityScenarioForComposable.activity.waitForComposeView() ) } }
- Run these Dropshots tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:connectedAndroidTest -Pdropshots.record
- Define your own annotation
Warning
Beware that Locale Strings in Preview Infos, unlike AndroidUiTestingUtils, use The BCP-47 tag but with + instead of - as separators, and have the prefix b+. Therefore, the BCP-47 tag "zh-Hans-CN" would be written as "b+zh+Hans+CN" instead. So for this case, you'd have to convert locale "b+zh+Hans+CN" to "zh-Hans-CN" in order to use it with AndroidUiTestingUtils
ComposablePreviewScanner also provides a class to customize the name of the generated screenshots based on its Preview Info. By default, it does not include the Preview Info in the screenshot file name if it is the same as its default value, but it can be configured to behave differently. That means, for @Preview(showBackground = false), showBackground would not be included in the screenshot file name since it is the default.
fun createScreenshotIdFor(preview: ComposablePreview<AndroidPreviewInfo>) =
AndroidPreviewScreenshotIdBuilder(preview)
// Paparazzi screenshot names already include className and methodName
// so ignore them to avoid them duplicated what might throw a FileNotFoundException
// due to the longName
.ignoreClassName()
.ignoreMethodName()
.ignoreForId("heightDp")
.ignoreForId("widthDp")
.overrideDefaultIdFor(
previewInfoName = "showBackground",
applyInfoValue = {
when (it.showBackground) {
true -> "WITH_BACKGROUND"
false -> "WITHOUT_BACKGROUND"
}
}
)
.build()
and then in your test
@Test
fun snapshot() {
paparazzi.snapshot(
name = createScreenshotIdFor(preview)
) {
preview()
}
}
So, for the following Preview
class MyClass {
@Preview(widthDp = 33, heightDp = 33, fontScale = 1.5f)
@Composable
fun MyComposable(){
...
}
}
createScreenshotIdFor(preview) will generate the following id: "MyClass.MyComposable.FONT_1_5f_WITHOUT_BACKGROUND"
Since 0.4.0, ComposablePreviewScanner also provides DevicePreviewInfoParser.parse(device: String)
which returns a Device
object containing all the necessary information to support different devices in your Roborazzi & Paparazzi screenshot tests!
It can parse ALL possible combinations of "device strings" up to Android Studio Lady Bug, namely:
// The over 80 devices supported either by id and/or name, for instance:
@Preview(device = "id:pixel_9_pro")
@Preview(device = "name:Pixel 9 Pro")
@Preview(device = "spec:parent=pixel_9_pro, orientation=landscape, navigation=buttons")
// And custom devices
@Preview(device = "spec:width = 411dp, height = 891dp, orientation = landscape, dpi = 420, isRound = false, chinSize = 0dp, cutout = corner")
@Preview(device = "spec:id=reference_desktop,shape=Normal,width=1920,height=1080,unit=px,dpi=160") // in pixels
@Preview(device = "spec:id=reference_desktop,shape=Normal,width=1920,height=1080,unit=dp,dpi=160") // in dp
...
If you are using Roborazzi, you can streamline the process by just calling
RobolectricDeviceQualifierBuilder.build(preview.previewInfo)?.run {
RuntimeEnvironment.setQualifiers(this)
}
before setting any other Robolectric 'cumulative qualifier' in your tests.
For further info on how to use them, see Roborazzi and Paparazzi sections respectively.
This library is written on top of ClassGraph, an uber-fast parallelized classpath scanner.
Classpath can scan everything that is available either at bytecode level or at runtime.
This is also the case of annotations without retention or with either AnnotationRetention.BINARY
or AnnotationRetention.RUNTIME
, like Android Composable Previews
package androidx.compose.ui.tooling.preview
...
@Retention(AnnotationRetention.BINARY)
annotation class Preview(
...
)
However, those with AnnotationRetention.SOURCE
are not visible to Classgraph. Such annotations are mainly used for IDE tooling, and that is the case for the Compose-Desktop Preview annotation.
package androidx.compose.desktop.ui.tooling.preview
...
@Retention(AnnotationRetention.SOURCE)
annotation class Preview
Since Compose Multiplatform 1.6.0, Jetbrains has added support for @Preview
s in common
. ComposablePreviewScanner can also
scan such Previews when running on any jvm-target, like
- Android
- Desktop
- Jvm
ComposablePreviewScanner provides a CommonComposablePreviewScanner
for that purpose.
Assuming that you have:
- some Compose Multiplatform
@Previews
defined incommon
- some Android screenshot tests in place
Here is how you could also run screenshot tests for those Compose Multiplatform @Previews
together, for instance, with Roborazzi (would also work with Paparazzi or any Instrumentation-based library).
-
Add
:jvm
dependency from ComposablePreviewScanner 0.2.0+testImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
-
Add an additional Parameterized screenshot test for these Compose Multiplatform
@Previews
@RunWith(ParameterizedRobolectricTestRunner::class)
class PreviewParameterizedTests(
private val preview: ComposablePreview<CommonPreviewInfo>,
) {
companion object {
// Optimization: This avoids scanning for every test
private val cachedPreviews: List<ComposablePreview<CommonPreviewInfo>> by lazy {
CommonComposablePreviewScanner()
.scanPackageTrees("your.package", "your.package2") // those where your common @Previews are located
.includeAnnotationInfoForAllOf(RoborazziConfig::class.java)
.getPreviews()
}
@JvmStatic
@ParameterizedRobolectricTestRunner.Parameters
fun values(): List<ComposablePreview<CommonPreviewInfo>> = cachedPreviews
}
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [30]) // same as the previews we've filtered
@Test
fun snapshot() {
captureRoboImage(
roborazziOptions = RoborazziOptionsMapper.createFor(preview)
) {
preview()
}
}
}
- Run these screenshot tests together with the existing ones by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziDebug
As we've seen in the previous section How it works, Compose-Desktop previews are still not visible to ClassGraph since they use AnnotationRetention.SOURCE
.
There is already an open issue to change it to AnnotationRetention.BINARY
, which would allow ClassGraph to find them.
In the meanwhile, it is also possible to workaround this limitation with ComposablePreviewScanner as follows.
-
Add
:jvm
dependency from ComposablePreviewScanner 0.2.0+ and use Roborazzi, since it is the only Screenshot Testing Library that supports Compose-DesktoptestImplementation("com.github.sergio-sastre.ComposablePreviewScanner:jvm:<version>")
-
Configure Roborazzi as described in the corresponding "Multiplatform support" section
-
Define your own annotation. The only condition is not to define
AnnotationRetention.SOURCE
package my.package.path @Target(AnnotationTarget.FUNCTION) annotation class VisibleForScreenshotTesting
-
Annotate the Desktop Composables you want to generate screenshot tests for with this annotation, e.g.
class MyClass {
@VisibleForScreenshotTesting
@Preview // It'd also work without this annotation
@Composable
fun MyDesktopComposable() {
...
}
}
- Create the test. Unfortunately,
kotlin.test
doesn't provide any means for parameterized tests, so we'll need to iterate through all previews. That has the disadvantage that, if it fails to record/verify one of the preview screenshots, subsequent screenshots won't be recorded/verified (hard asserts).
class DesktopPreviewScreenshotTests {
@OptIn(ExperimentalTestApi::class)
@Test
fun desktopPreviews() {
ROBORAZZI_DEBUG = true
val listComposables = JvmAnnotationScanner(
annotationToScanClassName = "my.package.path.VisibleForScreenshotTesting"
)
.scanPackageTrees("my.package.tree1", "my.package.tree2")
.getPreviews()
listComposables.forEach { composablePreview ->
runDesktopComposeUiTest {
setContent {
composablePreview()
}
onRoot().captureRoboImage(
filePath = "$composablePreview.png", // optional, works without it too
)
}
}
}
}
- Run these Roborazzi tests by executing the corresponding command e.g. ./gradlew yourModule:recordRoborazziJvm (if using the Kotlin Jvm Plugin)
In these tech-talks have also been mentioned the benefits of using ComposablePreviewScanner:
- DroidKaigi 2024 [in JA π―π΅ with EN π¬π§ slides]:
Understand the mechanism! Let's do screenshots testing of Compose Previews with various variations by Sumio Toyama
Composable Preview Scanner logo modified from one by Freepik - Flaticon