Coroutines doesn't play well with JNLP security model

[NikolayMetchev]

jnlptest.zip
I have created and attached a simple swing app that uses JNLP and co-routines.
I have in this app code that uses both swingworker and co-routines to download the html from google.com.
It also does this using java.net.HttpURLConnection and apache httpclient.

The swingworker code when run through JNLP works just fine without any problems (as long as the jnlp is signed).

The co-routines code exhibits 2 different behaviours:

1. java.net.HttpURLConnection - results in warning dialogs poping up asking the user for permissions to connect to google
2. apache http: java.security.AccessControlException

Exception in thread "AWT-EventQueue-2" java.util.concurrent.CompletionException: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.version" "read")
at java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source)
at java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(Unknown Source)
at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.security.AccessControlException: access denied ("java.util.PropertyPermission" "java.version" "read")
at java.security.AccessControlContext.checkPermission(Unknown Source)
at java.security.AccessController.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkPermission(Unknown Source)
at com.sun.javaws.security.JavaWebStartSecurity.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkPropertyAccess(Unknown Source)
at java.lang.System.getProperty(Unknown Source)
at org.apache.http.util.VersionInfo.getUserAgent(VersionInfo.java:321)
at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:1046)
at org.apache.http.impl.client.HttpClients.createDefault(HttpClients.java:56)
at jnlp.MainKt.getHTMLWithApache(main.kt:88)
at jnlp.MainKt$main$4.invoke(main.kt:29)
at jnlp.MainKt$main$4.invoke(main.kt)
at jnlp.MainKt$coroutine$1$1$1.get(main.kt:38)
at jnlp.MainKt$coroutine$1$1$1.get(main.kt)
... 6 more

To reproduce you should be able to just unzip the directory and double click on build/jnlp/launch.jnlp

[elizarov]

Are you sure it is a problem with coroutines? I'm not seeing any kotlinx.coroutines code in your stack trace. I'm seeing a problem Apache HTTP. It looks like with JNLP you should sign all the libraries you are using with your own certificate and grant them all the necessarily permissions.

[NikolayMetchev]

Hello @elizarov
Unfortunately Java truncated the stack trace for some reason. At the bottom of the stack trace you have
"... 6 more" in there it should have kotlinx.coroutines classes.
All libraries are signed with a certificate. This is done using gradle and the "de.gliderpilot.jnlp" plugin.
You should find everything you need in the zip file.

[userquin]

hi:

Have yo try to execute http call (httpurlconnection) with AccessController.doProvileged???

Even if the JAR is signed, you must use AccessController.doPrivileged, JNLP or applet or in a sandbox with security manager enabled.

the link for AccessController

[NikolayMetchev]

@userquin I am aware of AccessController but I don't think that is a viable solution. The JNLP in question is signed and has the following tag which means it shouldn't need to request permissions:

<security>
    <all-permissions />
  </security> 

The SwingWorker version and coroutine version executes exactly the same bit of code, the difference being that coroutine code behaves differently.

[userquin]

try use this:

fun String.fetch(): String = AccessController.doPrivileged(PrivilegedAction<String> {
    URL(this).let {
        var httpURLConnection: HttpURLConnection? = null
        try {
            httpURLConnection = it.openConnection() as HttpURLConnection
            httpURLConnection.requestMethod = "GET"
            BufferedReader(InputStreamReader(httpURLConnection.inputStream)).use {
                it.readLines().joinToString("")
            }
        } catch (e: Exception) {
            throw RuntimeException(e)
        } finally {
            httpURLConnection?.disconnect()
        }
    }
})

[NikolayMetchev]

@userquin I know that will work, however I shouldn't have to do that. In our more complicated production app I had to add these calls all over the place where coroutines were used and in the end the application still didn't work because it ended up with a NullPointerException. I am looking for an explanation why in the example app that I have attached as part of this bug report the SwingWorker code works as expected and the coroutines code doesn't.

[userquin]

ok, the problem may be the pool you are using to execute the coroutine, maybe providing your own context using on it a security manager... I'll trying to find the code I use...

[userquin]

I have downloaded your zip and I have executed the jnlp from build\jnlp directory (enabling file:// in java control panel: Java 1.8) and HttpURLConnection works, Apache not working. The only difference from swing worker and coroutine is java web start prompts a confirm dialog to connect to google using coroutine (confirm in spanish):

[NikolayMetchev]

@userquin thanks for testing. You have just confirmed everything I have said in the original bug report. If you enable the JNLP console you will see the exception that happens with the apache version there.

[NikolayMetchev]

Also I should point out that you can open the project in Idea and view/edit the code to try and see if you can figure out why coroutines is breaking the JNLP security model.

[userquin]

As I mention before, the problem is the pool that Swing (launch(Swing)) context uses: it must delegate on security manage or at least check if one exists and then enqueue the pool in the same group the security manager runs.

The problem with its pool is that in its context there is no way to elevate required permissions, so you must create one that checks if a security manager.

One simple way is provide to your pool a ThreadFactory; then check if a security manager exists, if so, then create threads using the same group as the security manager and remeber to create the thread with a runnable that switchs the context class loader to the loader that creates de access controller context required to elevate the privileges.

In 5 minutes I provide an example...

[userquin]

Executors.privilegedThreadFactory method can be used to create a new FixedThreadPool (also from Executors class) providing number of threads and privilegedThreadFactory as threadfactory.

By default launch uses CommonPool:

public val DefaultDispatcher: CoroutineDispatcher = CommonPool
...
public fun launch(
    context: CoroutineContext = DefaultDispatcher,

For example, kotlinx.coroutines.experimental.CommonPool. uses this:

    private fun createPool(): ExecutorService {
        val fjpClass = Try { Class.forName("java.util.concurrent.ForkJoinPool") }
            ?: return createPlainPool()
        if (!usePrivatePool) {
            Try { fjpClass.getMethod("commonPool")?.invoke(null) as? ExecutorService }
                ?.let { return it }
        }
        Try { fjpClass.getConstructor(Int::class.java).newInstance(defaultParallelism()) as? ExecutorService }
            ?. let { return it }
        return createPlainPool()
    }

    private fun createPool(): ExecutorService {
        val threadId = AtomicInteger()
        return Executors.newFixedThreadPool(defaultParallelism()) {
            Thread(it, "CommonPool-worker-${threadId.incrementAndGet()}").apply { isDaemon = true }
        }
    }

Create a new CoroutineDispatcher like CommonPool and just change the way it creates the pool:

    private fun createPool(): ExecutorService {
        val threadId = AtomicInteger()
        return Executors.newFixedThreadPool(defaultParallelism(), Executors.privilegedThreadFactory()) {
            Thread(it, "PrivilegedCommonPool-worker-${threadId.incrementAndGet()}").apply { isDaemon = true }
        }
    }

and finally just call launch with the new one CoroutineDispatcher created (copy/paste CommonPool and change only createPool and remove createPlainPool: you will use this on desktop environment, so using fork join does not matter):

object PrivilegedCommonPool: CoroutineDispatcher() {
    private fun createPool(): ExecutorService {
        val threadId = AtomicInteger()
        return Executors.newFixedThreadPool(defaultParallelism(), Executors.privilegedThreadFactory()) {
            Thread(it, "PrivilegedCommonPool-worker-${threadId.incrementAndGet()}").apply { isDaemon = true }
        }
    }
}

define your dispatcher and your privileged dispatcher:

public val PrivilegedDispatcher: CoroutineDispatcher = PrivilegedCommonPool
...
public fun launchPrivileged(
    context: CoroutineContext = PrivilegedDispatcher,
    start: CoroutineStart = CoroutineStart.DEFAULT,
    block: suspend CoroutineScope.() -> Unit
): Job {
    val newContext = newCoroutineContext(context)
    val coroutine = if (start.isLazy)
        LazyStandaloneCoroutine(newContext, block) else
        StandaloneCoroutine(newContext, active = true)
    coroutine.initParentJob(context[Job])
    start(block, coroutine, coroutine)
    return coroutine
}

and use it:

launchPrivileged({
      showResult(jFrame, CompletableFuture.supplyAsync {
        getHtml()
      }.await())
})

or you just name it launch and import your launch instead kotlin, the code will be the same, just change 1 or 2 imports in the file...

[userquin]

ok, cannot be done, we must use DefaultExecutor it is internal... We must request this new feature to kotlin:

@elizarov can this be included in kotlin coroutines (or in swing or jnlp extension)?

Include in CoroutineContext.kt

/**
 * This is the privileged [CoroutineDispatcher] that is used by all standard builders like
 * [launch], [async], etc if no dispatcher nor any other [ContinuationInterceptor] is specified in their context.
 *
 * It is currently equal to [PrivilegedCommonPool], but the value is subject to change in the future.
 */
public val PrivilegedDispatcher: CoroutineDispatcher = PrivilegedCommonPool

New PrivilegedCommonPool.kt

package kotlinx.coroutines.experimental

import java.util.concurrent.*
import kotlin.coroutines.experimental.CoroutineContext

object PrivilegedCommonPool: CoroutineDispatcher() {

    @Volatile
    private var _pool: Executor? = null

    private fun createPool(): ExecutorService {
        return Executors.newFixedThreadPool(defaultParallelism(), Executors.privilegedThreadFactory())
    }

    private fun defaultParallelism() = (Runtime.getRuntime().availableProcessors() - 1).coerceAtLeast(1)

    @Synchronized
    private fun getOrCreatePoolSync(): Executor =
            _pool
                    ?: createPool().also { _pool = it }

    override fun dispatch(context: CoroutineContext, block: Runnable) =
            try { (PrivilegedCommonPool._pool ?: PrivilegedCommonPool.getOrCreatePoolSync()).execute(timeSource.trackTask(block)) }
            catch (e: RejectedExecutionException) {
                timeSource.unTrackTask()
                DefaultExecutor.execute(block)
            }

    // used for tests
    @Synchronized
    internal fun usePrivatePool() {
        shutdown(0)
        PrivilegedCommonPool._pool = null
    }

    // used for tests
    @Synchronized
    internal fun shutdown(timeout: Long) {
        (PrivilegedCommonPool._pool as? ExecutorService)?.apply {
            shutdown()
            if (timeout > 0)
                awaitTermination(timeout, TimeUnit.MILLISECONDS)
            shutdownNow().forEach { DefaultExecutor.execute(it) }
        }
        PrivilegedCommonPool._pool = Executor { throw RejectedExecutionException("PrivilegedCommonPoolwas shutdown") }
    }

    // used for tests
    @Synchronized
    internal fun restore() {
        shutdown(0)
        PrivilegedCommonPool._pool = null
    }

    override fun toString(): String = "PrivilegedCommonPool"
}

with these changes we can use launch(PrivilegedCommonPool) {...} when running on a sandbox (SecurityManager) solving the problem.

by the way, how can I test this on intellij?