Introduce Native IR transformations (#363)

Native IR transformations are now available: compiler plugin implements atomic operations via K/N stdlib atomic intrinsics. Atomic arrays, delegated properties and traces are supported by the compiler plugin as well.
To enable Native IR transformations, set the flag `kotlinx.atomicfu.enableNativeIrTransformations=true` in the `gradle.properties` file.

Author: mvicsokolova

README.md

@@ -9,7 +9,7 @@
 >We do provide a compatibility of atomicfu-transformed artifacts between releases, but we do not provide 
 >strict compatibility guarantees on plugin API and its general stability between Kotlin versions.
 
-**Atomicfu** is a multiplatform library that provides the idiomatic and effective way of using atomic operations in Kotlin.
+**Atomicfu** is a multiplatform library that provides the idiomatic and efficient way of using atomic operations in Kotlin.
 
 ## Table of contents
 - [Requirements](#requirements)
@@ -46,7 +46,8 @@ Starting from version `0.22.0` of the library your project is required to use:
 * Code it like a boxed value `atomic(0)`, but run it in production efficiently:
   * For **JVM**: an atomic value is represented as a plain value atomically updated with `java.util.concurrent.atomic.AtomicXxxFieldUpdater` from the Java standard library.
   * For **JS**: an atomic value is represented as a plain value.
-  * For **Native** and **Wasm**: an atomic value is not transformed, it remains boxed, and `kotlinx-atomicfu` library is used as a runtime dependency.
+  * For **Native**: atomic operations are delegated to Kotlin/Native atomic intrinsics.
+  * For **Wasm**: an atomic value is not transformed, it remains boxed, and `kotlinx-atomicfu` library is used as a runtime dependency.
 * Use Kotlin-specific extensions (e.g. inline `loop`, `update`, `updateAndGet` functions).
 * Use atomic arrays, user-defined extensions on atomics and locks (see [more features](#more-features)).
 * [Tracing operations](#tracing-operations) for debugging.
@@ -247,17 +248,13 @@ public var foo: T by _foo            // public delegated property (val/var)
   (more specifically, `complex_expression` should not have branches in its compiled representation).
   Extract `complex_expression` into a variable when needed.
 
-## Transformation modes
+## Atomicfu compiler plugin
 
-Basically, Atomicfu library provides an effective usage of atomic values by performing the transformations of the compiled code.
-For JVM and JS there 2 transformation modes available: 
-* **Post-compilation transformation** that modifies the compiled bytecode or `*.js` files. 
-* **IR transformation** that is performed by the atomicfu compiler plugin.
-
-### Atomicfu compiler plugin
-
-Compiler plugin transformation is less fragile than transformation of the compiled sources 
-as it depends on the compiler IR tree.
+To provide a user-friendly atomic API on the frontend and efficient usage of atomic values on the backend kotlinx-atomicfu library uses the compiler plugin to transform 
+IR for all the target backends: 
+* **JVM**: atomics are replaced with `java.util.concurrent.atomic.AtomicXxxFieldUpdater`.
+* **Native**: atomics are implemented via atomic intrinsics on Kotlin/Native.
+* **JS**: atomics are unboxed and represented as plain values.
 
 To turn on IR transformation set these properties in your `gradle.properties` file:
 
@@ -266,6 +263,7 @@ To turn on IR transformation set these properties in your `gradle.properties` fi
 
 ```groovy
 kotlinx.atomicfu.enableJvmIrTransformation=true // for JVM IR transformation
+kotlinx.atomicfu.enableNativeIrTransformation=true // for Native IR transformation
 kotlinx.atomicfu.enableJsIrTransformation=true // for JS IR transformation
 ```
 

atomicfu-gradle-plugin/src/main/kotlin/kotlinx/atomicfu/plugin/gradle/AtomicFUGradlePlugin.kt

@@ -35,6 +35,7 @@ private const val TEST_IMPLEMENTATION_CONFIGURATION = "testImplementation"
 private const val ENABLE_JS_IR_TRANSFORMATION_LEGACY = "kotlinx.atomicfu.enableIrTransformation"
 private const val ENABLE_JS_IR_TRANSFORMATION = "kotlinx.atomicfu.enableJsIrTransformation"
 private const val ENABLE_JVM_IR_TRANSFORMATION = "kotlinx.atomicfu.enableJvmIrTransformation"
+private const val ENABLE_NATIVE_IR_TRANSFORMATION = "kotlinx.atomicfu.enableNativeIrTransformation"
 private const val MIN_SUPPORTED_GRADLE_VERSION = "7.0"
 private const val MIN_SUPPORTED_KGP_VERSION = "1.7.0"
 
@@ -78,6 +79,7 @@ private fun Project.applyAtomicfuCompilerPlugin() {
         extensions.getByType(AtomicfuKotlinGradleSubplugin.AtomicfuKotlinGradleExtension::class.java).apply {
             isJsIrTransformationEnabled = rootProject.getBooleanProperty(ENABLE_JS_IR_TRANSFORMATION)
             isJvmIrTransformationEnabled = rootProject.getBooleanProperty(ENABLE_JVM_IR_TRANSFORMATION)
+            isNativeIrTransformationEnabled = rootProject.getBooleanProperty(ENABLE_NATIVE_IR_TRANSFORMATION)
         }
     } else {
         // for KGP >= 1.6.20 && KGP <= 1.7.20:
@@ -171,6 +173,11 @@ private fun Project.needsJvmIrTransformation(target: KotlinTarget): Boolean =
     rootProject.getBooleanProperty(ENABLE_JVM_IR_TRANSFORMATION) &&
             (target.platformType == KotlinPlatformType.jvm || target.platformType == KotlinPlatformType.androidJvm)
 
+private fun Project.needsNativeIrTransformation(target: KotlinTarget): Boolean =
+    rootProject.getBooleanProperty(ENABLE_NATIVE_IR_TRANSFORMATION) &&
+            (target.platformType == KotlinPlatformType.native)
+
+
 private fun KotlinTarget.isJsIrTarget() =
     (this is KotlinJsTarget && this.irTarget != null) ||
             (this is KotlinJsIrTarget && this.platformType != KotlinPlatformType.wasm)
@@ -179,7 +186,8 @@ private fun Project.isTransformationDisabled(target: KotlinTarget): Boolean {
     val platformType = target.platformType
     return !config.transformJvm && (platformType == KotlinPlatformType.jvm || platformType == KotlinPlatformType.androidJvm) ||
             !config.transformJs && platformType == KotlinPlatformType.js ||
-            platformType == KotlinPlatformType.wasm
+            platformType == KotlinPlatformType.wasm ||
+            !needsNativeIrTransformation(target) && platformType == KotlinPlatformType.native
 }
 
 // Adds kotlinx-atomicfu-runtime as an implementation dependency to the JS IR target:
@@ -280,20 +288,29 @@ private fun Project.configureTasks() {
 
 private fun Project.configureJvmTransformation() {
     if (kotlinExtension is KotlinJvmProjectExtension || kotlinExtension is KotlinAndroidProjectExtension) {
-        configureTransformationForTarget((kotlinExtension as KotlinSingleTargetExtension<*>).target)
+        val target = (kotlinExtension as KotlinSingleTargetExtension<*>).target
+        if (!needsJvmIrTransformation(target)) {
+            configureTransformationForTarget(target)   
+        }
     }
 }
 
-private fun Project.configureJsTransformation() =
-    configureTransformationForTarget((kotlinExtension as KotlinJsProjectExtension).js())
+private fun Project.configureJsTransformation() {
+    val target = (kotlinExtension as KotlinJsProjectExtension).js()
+    if (!needsJsIrTransformation(target)) {
+        configureTransformationForTarget(target)
+    }
+}
 
 private fun Project.configureMultiplatformTransformation() =
     withKotlinTargets { target ->
+        // Skip transformation for common, native and wasm targets or in case IR transformation by the compiler plugin is enabled (for JVM or JS targets)
         if (target.platformType == KotlinPlatformType.common || 
             target.platformType == KotlinPlatformType.native ||
-            target.platformType == KotlinPlatformType.wasm
+            target.platformType == KotlinPlatformType.wasm ||
+            needsJvmIrTransformation(target) || needsJsIrTransformation(target)
            ) {
-            return@withKotlinTargets // skip creation of transformation task for common, native and wasm targets
+            return@withKotlinTargets
         }
         configureTransformationForTarget(target)
     }
@@ -302,8 +319,6 @@ private fun Project.configureTransformationForTarget(target: KotlinTarget) {
     val originalDirsByCompilation = hashMapOf<KotlinCompilation<*>, FileCollection>()
     val config = config
     target.compilations.all compilations@{ compilation ->
-        // do not modify directories if compiler plugin is applied
-        if (needsJvmIrTransformation(target) || needsJsIrTransformation(target)) return@compilations
         val compilationType = compilation.name.compilationNameToType()
             ?: return@compilations // skip unknown compilations
         val classesDirs = compilation.output.classesDirs
@@ -329,7 +344,7 @@ private fun Project.configureTransformationForTarget(target: KotlinTarget) {
         val transformTask = when (target.platformType) {
             KotlinPlatformType.jvm, KotlinPlatformType.androidJvm -> {
                 // create transformation task only if transformation is required and JVM IR compiler transformation is not enabled
-                if (config.transformJvm && !rootProject.getBooleanProperty(ENABLE_JVM_IR_TRANSFORMATION)) {
+                if (config.transformJvm) {
                     project.registerJvmTransformTask(compilation)
                         .configureJvmTask(
                             compilation.compileDependencyFiles,

integration-testing/src/functionalTest/kotlin/kotlinx.atomicfu.gradle.plugin.test/cases/MppProjectTest.kt

@@ -48,4 +48,22 @@ class MppProjectTest {
         mppSample.checkMppWasmJsImplementationDependencies()
         mppSample.checkMppWasmWasiImplementationDependencies()
     }
+
+    @Test
+    fun testMppNativeWithEnabledIrTransformation() {
+        mppSample.enableNativeIrTransformation = true
+        assertTrue(mppSample.cleanAndBuild().isSuccessful)
+        mppSample.checkMppNativeCompileOnlyDependencies()
+        // TODO: klib checks are skipped for now because of this problem KT-61143
+        //mppSample.buildAndCheckNativeKlib()
+    }
+
+    @Test
+    fun testMppNativeWithDisabledIrTransformation() {
+        mppSample.enableNativeIrTransformation = false
+        assertTrue(mppSample.cleanAndBuild().isSuccessful)
+        mppSample.checkMppNativeImplementationDependencies()
+        // TODO: klib checks are skipped for now because of this problem KT-61143
+        //mppSample.buildAndCheckNativeKlib()
+    }
 }

integration-testing/src/functionalTest/kotlin/kotlinx.atomicfu.gradle.plugin.test/framework/checker/ArtifactChecker.kt

@@ -8,14 +8,15 @@ import kotlinx.atomicfu.gradle.plugin.test.framework.runner.GradleBuild
 import kotlinx.atomicfu.gradle.plugin.test.framework.runner.cleanAndBuild
 import org.objectweb.asm.*
 import java.io.File
+import java.net.URLClassLoader
 import kotlin.test.assertFalse
 
 internal abstract class ArtifactChecker(private val targetDir: File) {
 
     private val ATOMIC_FU_REF = "Lkotlinx/atomicfu/".toByteArray()
     protected val KOTLIN_METADATA_DESC = "Lkotlin/Metadata;"
 
-    private val projectName = targetDir.name.substringBeforeLast("-")
+    protected val projectName = targetDir.name.substringBeforeLast("-")
 
     val buildDir
         get() = targetDir.resolve("build").also {
@@ -60,8 +61,63 @@ private class BytecodeChecker(targetDir: File) : ArtifactChecker(targetDir) {
     }
 }
 
+private class KlibChecker(targetDir: File) : ArtifactChecker(targetDir) {
+
+    val nativeJar = System.getProperty("kotlin.native.jar")
+
+    val classLoader: ClassLoader = URLClassLoader(arrayOf(File(nativeJar).toURI().toURL()), this.javaClass.classLoader)
+
+    private fun invokeKlibTool(
+        kotlinNativeClassLoader: ClassLoader?,
+        klibFile: File,
+        functionName: String,
+        hasOutput: Boolean,
+        vararg args: Any
+    ): String {
+        val libraryClass = Class.forName("org.jetbrains.kotlin.cli.klib.Library", true, kotlinNativeClassLoader)
+        val entryPoint = libraryClass.declaredMethods.single { it.name == functionName }
+        val lib = libraryClass.getDeclaredConstructor(String::class.java, String::class.java, String::class.java)
+            .newInstance(klibFile.canonicalPath, null, "host")
+
+        val output = StringBuilder()
+
+        // This is a hack. It would be better to get entryPoint properly
+        if (args.isNotEmpty()) {
+            entryPoint.invoke(lib, output, *args)
+        } else if (hasOutput) {
+            entryPoint.invoke(lib, output)
+        } else {
+            entryPoint.invoke(lib)
+        }
+        return output.toString()
+    }
+
+    override fun checkReferences() {
+        val classesDir = buildDir.resolve("classes/kotlin/")
+        if (classesDir.exists() && classesDir.isDirectory) {
+            classesDir.walkBottomUp().singleOrNull { it.isFile && it.name == "$projectName.klib" }?.let { klib ->
+                val klibIr = invokeKlibTool(
+                    kotlinNativeClassLoader = classLoader,
+                    klibFile = klib,
+                    functionName = "ir",
+                    hasOutput = true,
+                    false
+                )
+                assertFalse(klibIr.toByteArray().findAtomicfuRef(), "Found kotlinx/atomicfu in klib ${klib.path}:\n $klibIr")
+            } ?: error(" Native klib $projectName.klib is not found in $classesDir")
+        }
+    }
+}
+
 internal fun GradleBuild.buildAndCheckBytecode() {
     val buildResult = cleanAndBuild()
     require(buildResult.isSuccessful) { "Build of the project $projectName failed:\n ${buildResult.output}" }
     BytecodeChecker(this.targetDir).checkReferences()
 }
+
+// TODO: klib checks are skipped for now because of this problem KT-61143
+internal fun GradleBuild.buildAndCheckNativeKlib() {
+    val buildResult = cleanAndBuild()
+    require(buildResult.isSuccessful) { "Build of the project $projectName failed:\n ${buildResult.output}" }
+    KlibChecker(this.targetDir).checkReferences()
+}

integration-testing/src/functionalTest/kotlin/kotlinx.atomicfu.gradle.plugin.test/framework/checker/DependenciesChecker.kt

@@ -59,6 +59,19 @@ internal fun GradleBuild.checkMppWasmWasiImplementationDependencies() {
     checkAtomicfuDependencyIsPresent(listOf("wasmWasiCompileClasspath", "wasmWasiRuntimeClasspath"), commonAtomicfuDependency)
 }
 
+// Checks Native target of an MPP project
+internal fun GradleBuild.checkMppNativeCompileOnlyDependencies() {
+    // Here the name of the native target is hardcoded because the tested mpp-sample project declares this target and
+    // KGP generates the same set of dependencies for every declared native target ([mingwX64|linuxX64|macosX64...]CompileKlibraries)
+    checkAtomicfuDependencyIsPresent(listOf("macosX64CompileKlibraries"), commonAtomicfuDependency)
+    checkAtomicfuDependencyIsAbsent(listOf("macosX64MainImplementation"), commonAtomicfuDependency)
+}
+
+// Checks Native target of an MPP project
+internal fun GradleBuild.checkMppNativeImplementationDependencies() {
+    checkAtomicfuDependencyIsPresent(listOf("macosX64CompileKlibraries", "macosX64MainImplementation"), commonAtomicfuDependency)
+}
+
 // Some dependencies may be not resolvable but consumable and will not be present in the output of :dependencies task,
 // in this case we should check .pom or .module file of the published project.
 // This method checks if the .module file in the sample project publication contains org.jetbrains.kotlinx:atomicfu dependency included.