Code Coverage not reported for instrumented tests (Android library module) on AGP 7.0

[raquezha]

Hi I'm trying this on my test multi-module project before trying this on the other project I'm working on. I've followed the steps and run ./gradlew rootCoveragePlugin but got this report:

--
com.raquezha.heograpiya.splash is 0% but when I run it in Android Studio local coverage it recognize the tests:

--

here's the file structure
as you can see Android Studio recognizes the local code coverage of splash module and says its 100% lines covered

here's my top-level build.gradle

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    apply from: rootProject.file('dependencies.gradle')

    repositories {
        google()
        mavenCentral()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:7.0.0-alpha15'
        classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.0'
        classpath 'com.google.gms:google-services:4.3.5'
        classpath 'nl.neotech.plugin:android-root-coverage-plugin:1.4.0'

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

plugins {
    id "nl.neotech.plugin.rootcoverage" version "1.4.0"
}

apply plugin: 'nl.neotech.plugin.rootcoverage'

jacoco {
    toolVersion = "0.8.7"
}

tasks.withType(Test) {
    useJUnitPlatform()
    testLogging {
        exceptionFormat "full"
        events "started", "skipped", "passed", "failed"
        showStandardStreams true
    }
}

subprojects {
    afterEvaluate { project ->
        if (project.hasProperty('android')) {
            android {
                compileSdk 30
                buildToolsVersion "30.0.3"

                defaultConfig {
                    minSdk 26
                    targetSdk 30
                    versionCode 1
                    versionName "1.0"

                    testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
                }

                buildTypes {
                    release {
                        minifyEnabled false
                        proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
                    }

                    debug {
                        testCoverageEnabled true
                    }
                }

                compileOptions {
                    sourceCompatibility JavaVersion.VERSION_1_8
                    targetCompatibility JavaVersion.VERSION_1_8
                }

                kotlinOptions {
                    jvmTarget = '1.8'
                }

                buildFeatures {
                    viewBinding true
                }

                sourceSets {
                    main {
                        java.srcDirs = ['src/main/kotlin']
                    }
                    androidTest {
                        java.srcDirs = ['src/androidTest/kotlin']
                    }
                    test {
                        java.srcDirs = ['src/test/kotlin']
                    }
                }
            }
        }
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

here's the splash module build.gradle:

plugins {
    id 'com.android.library'
    id 'kotlin-android'
}

dependencies {
    implementation _libs.lottie

    implementation project(':shared')

    testImplementation project(path: ':shared', configuration: 'testDependencies')
    androidTestImplementation project(path: ':shared', configuration: 'androidTestDependencies')
}

tasks.withType(Test) {
    useJUnitPlatform()
    testLogging {
        exceptionFormat "full"
        events "started", "skipped", "passed", "failed"
        showStandardStreams true
    }
}

I'm using:

1. Android Studio Arctic Fox | 2020.3.1 Canary 15
2. Gradle JDK: AdoptOpenj9-11
3. gradle-wrapper: distributionUrl=https://services.gradle.org/distributions/gradle-7.0-bin.zip

I'm using canary because this is only a test project I want to test new stuff

[Rolf-Smit]

I'm not exactly sure what causes this, but I suspect it has something to do with using:

subprojects {
    afterEvaluate { project ->
        // Modifying srcDirs here
    }
}

It could be that the plugin gets configured before you configure the new srcDirs, meaning that the plugin will not see these? You could try to move the plugin apply block to after this block. But this is just me guessing.

Can you share this project with me so I can do some testing?

[raquezha]

Hi thanks for helping! you can check the project here: https://github.com/raquezha/heograpiya

[raquezha]

I've tried applying the apply block after subProjects{ } block but it did not work still 0%

[Rolf-Smit]

Hi @raquezha took me a while to take a look at this, but I found a few things:

Looking at the report that is generated, it is missing all execution data, which is needed to calculate coverage. You can check this for your self by going to the sessions part of the report:

This can have multiple causes, but in this case this happens because the Android Gradle Plugin updated where these files are located since version 7.0. This means the Plugin needs a small update.

However I also spotted this, which will also prevent coverage (even if I update the plugin):

In the root project build.gradle file you have this line:

getExecutionData().setFrom(fileTree(buildDir).include("/jacoco/*.exec"))

This basically overrides any of the default set execution data location's with the location you provided. Your location only looks at the buildDir of the root project, but normally execution data is generated in all the different module buildDir's, so this will definitely not work if you ask me.

TLDR: This plugin needs an update due to changes in the Android Gradle Plugin 7.0. I'm going to take care of this. (see update this is no longer something I can fix)

[raquezha]

Thank you! looking forward for the fix, If you need someone to test, I'll gladly help!

[Rolf-Smit]

Update:
I've done some work on making the plugin work with AGP 7.0. The integration test for this plugin now succeeds (which it didn't before because some coverage was missing).

However, on your specific project, there is still coverage missing. So I'm investigating what exactly causes that.

[Rolf-Smit]

Update 2:

Kotlin 1.5 is the issue?

Did some more testing and I found that even though there is a small change in AGP 7.0 (they moved the location of the .exec for unit tests to a different location). After fixing that there is still no unit-test coverage (instrumented unit-tests work fine).

I found that the generated .exec file for unit-tests is basically empty, which is weird.

When I downgrade Kotlin to version 1.4.x everything works as expected, the ` In other words, it seems this is not specifically a AGP 7.0 issue, but more likely a JaCoCo or Kotlin issue. On the other hand AGP 7.0 itself depends on Kotlin 1.4, so maybe that is what's causing the issue?

I also tried upgrading to Gradle 7.1, JaCoCo 8.8 (snapshot), Kotlin 1.5.10. But that didn't work.

For now I only know that downgrading to Kotlin 1.4, seems to work (as a workaround).

Coverage in Android Studio

Now the reason you are seeing coverage in Android Studio is because you are probably running the Intellij code coverage runner, and not the Jacoco coverage runner. If you switch to Jacoco you will notice that Android Studio will not report any coverage.

I will try to search further but as of now I don't have any idea's left. Maybe I should file a bug? But where? JaCoCo? Kotlin?

[kherink]

hi, is there any progress on the kotlin 1.5 coverage issue? if help is needed let me know.