Regression with Gradle's configuration cache in 0.9.15

[wilkinsona]

Describe the bug
In some circumstances, a Gradle build fails when run using --configuration-cache due to a Configuration instance that's captured by a lambda in NativeImpagePlugin. The same build works with 0.9.14.

To Reproduce
Sorry, I've yet to manage to boil things down into a minimal reproducer. Hopefully the configuration cache problem report (see below) will be sufficient to diagnose the problem.

Expected behavior
The Gradle plugin should work with --configuration-cache.

Logs

⌄ cannot serialize object of type org.gradle.api.internal.artifacts.configurations.DefaultConfiguration, a subtype of org.gradle.api.artifacts.Configuration, as these are not supported with the configuration cache. ?
  ⌄ fieldarg$1 of org.graalvm.buildtools.gradle.NativeImagePlugin$$Lambda$1852/0x0000000801a9f000
    ⌄ bean of type org.graalvm.buildtools.gradle.NativeImagePlugin$$Lambda$1852/0x0000000801a9f000
      ⌄ fieldtransformer of org.gradle.api.internal.provider.TransformBackedProvider
        ⌄ bean of type org.gradle.api.internal.provider.TransformBackedProvider
          ⌄ fieldprovider of org.gradle.configurationcache.serialization.codecs.ProviderBackedFileCollectionSpec
            ⌄ bean of type org.gradle.configurationcache.serialization.codecs.ProviderBackedFileCollectionSpec
              ⌄ field__configurationFileDirectories__ of org.graalvm.buildtools.gradle.internal.BaseNativeImageOptions
                ⌄ bean of type org.graalvm.buildtools.gradle.internal.BaseNativeImageOptions
                  ⌄ field_gr_owner_ of org.graalvm.buildtools.gradle.dsl.NativeResourcesOptions
                    ⌄ bean of type org.graalvm.buildtools.gradle.dsl.NativeResourcesOptions
                      ⌄ field__options__ of org.graalvm.buildtools.gradle.tasks.GenerateResourcesConfigFile
                        task:generateResourcesConfigFile of type org.graalvm.buildtools.gradle.tasks.GenerateResourcesConfigFile

System Info (please complete the following information):

• OS: macOS
• GraalVM Version: 22.2
• Java Version: 17
• Plugin version: 0.9.15

[melix]

@wilkinsona would it be possible for you to try to build and test this branch?

[wilkinsona]

That was quick. Thank you, @melix. That's better as the configuration cache no longer reports a problem. Unfortunately, the build still fails:

./gradlew --configuration-cache generateResourcesConfigFile
Configuration cache is an incubating feature.
Calculating task graph as no configuration cache is available for tasks: generateResourcesConfigFile

0 problems were found storing the configuration cache.

See the complete report at file:///Users/awilkinson/dev/temp/nbt-configuration-cache-problem/build/reports/configuration-cache/bub752nr41ig1gyl5xvezwkms/cku5jhqpzmyxpzegunrchhv9/configuration-cache-report.html

FAILURE: Build failed with an exception.

* What went wrong:
Configuration cache state could not be cached: field '__options__' from type 'org.graalvm.buildtools.gradle.tasks.GenerateResourcesConfigFile': error writing value of type 'org.gradle.api.internal.provider.DefaultProperty'
> Configuration cache state could not be cached: field '_gr_owner_' from type 'org.graalvm.buildtools.gradle.dsl.NativeResourcesOptions': error writing value of type 'org.graalvm.buildtools.gradle.internal.BaseNativeImageOptions'
   > Configuration cache state could not be cached: field '__configurationFileDirectories__' from type 'org.graalvm.buildtools.gradle.internal.BaseNativeImageOptions': error writing value of type 'org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection'
      > Configuration cache state could not be cached: field 'provider' from type 'org.gradle.configurationcache.serialization.codecs.ProviderBackedFileCollectionSpec': error writing value of type 'org.gradle.api.internal.provider.FlatMapProvider'
         > Configuration cache state could not be cached: field 'transformer' from type 'org.gradle.api.internal.provider.TransformBackedProvider': error writing value of type 'org.graalvm.buildtools.gradle.NativeImagePlugin$$Lambda$1256/0x000000080155dc28'
            > Configuration cache state could not be cached: field 'capturedArgs' from type 'java.lang.invoke.SerializedLambda': error writing value of type '[Ljava.lang.Object;'
               > Unable to make field private final java.util.function.BiConsumer java.util.stream.Collectors$CollectorImpl.accumulator accessible: module java.base does not "opens java.util.stream" to unnamed module @bbf12e9

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
Configuration cache entry discarded.

I don't see the failure with 0.9.14. It occurs with 0.9.15 and 0.9.16-SNAPSHOT.

nbt-configuration-cache-problem.zip contains a reproducer that should fail as above.

[melix]

Thanks for the reproducer! 👀

[melix]

It seems we're actually hitting a Gradle bug here :/ I'm trying to see if we can workaround.

[melix]

How critical is this for you? I'm afraid I'm hitting wall after wall, there seem to be bugs in Gradle I can't workaround :(

Latest problem is:

Caused by: java.io.InvalidObjectException: ReflectiveOperationException during deserialization
        ... 168 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        ... 168 more
Caused by: java.lang.ClassCastException: class org.gradle.api.internal.file.DefaultFileCollectionFactory$ResolvingFileCollection cannot be cast to class org.gradle.api.artifacts.Configuration (org.gradle.api.internal.file.DefaultFileCollectionFactory$ResolvingFileCollection and org.gradle.api.artifacts.Configuration are in unnamed module of loader org.gradle.internal.classloader.VisitableURLClassLoader @4cdf35a9)
        at org.graalvm.buildtools.gradle.NativeImagePlugin.$renamedDeserializeLambda$(NativeImagePlugin.java:154)
        at org.graalvm.buildtools.gradle.NativeImagePlugin.$deserializeLambda$(NativeImagePlugin.java)
        ... 171 more

Happening when the cache is deserialized.

[wilkinsona]

Thanks for trying, @melix. It's not hugely critical. We have some users who care about the configuration cache but they're definitely a minority. I've disabled the failing test that alerted me to the problem so we're not blocked. Let's revisit this when there's more time to do so.

[shoaniki]

This isn't a showstopper by any means, but it is causing a certain amount of pain -- we've recently put some effort into fixing our build to support the configuration cache, and this is one of the few issues preventing us turning it on globally.

Just realised as I was typing this that #339 hasn't been merged yet – is it worth trying with that patch, or are the Gradle problems more deep-rooted than that?

[melix]

is it worth trying with that patch, or are the Gradle problems more deep-rooted than that?

The patch isn't enough. There are other problems arising which have no solution yet in Gradle.

[shoaniki]

That's disappointing, but good to know, thank you.
(Are there any relevant Gradle issues I can follow?)

[melix]

I took another stab at it and I think I fixed it in #339, but it will require bumping the minimal version of Gradle to 7.4 (for the plugin, independently of the config cache) and 7.5 (with the configuration cache).