jdk16 extended.perf renaissance-als_0 java.lang.reflect.InaccessibleObjectException #2253

lumpfish · 2021-02-10T12:45:10Z

jdk16 extended.perf test target renaissance-als_0 fails with java.lang.reflect.InaccessibleObjectException

Looks like the benchmark is trying to access a module which is no longer available.

The test fails in the same way on both hotspot and openj9 builds.

Test output:

07:19:14  ===============================================
07:19:15  Running test renaissance-als_0 ...
07:19:15  ===============================================
07:19:15  renaissance-als_0 Start Time: Sun Feb  7 07:19:14 2021 Epoch Time (ms): 1612682354210
07:19:15  Nothing to be done for setup.
07:19:15  variation: NoOptions
07:19:16  JVM_OPTIONS:  
07:19:16  { itercnt=1; \
07:19:16  mkdir -p "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/\\TKG\\output_16126822706614\\renaissance-als_0"; \
07:19:16  cd "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/\\TKG\\output_16126822706614\\renaissance-als_0"; \
07:19:16  "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdkbinary/j2sdk-image\\bin\\java" -jar "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests///..//jvmtest\\perf\\renaissance\\renaissance-mit.jar" --json ""C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/\\TKG\\output_16126822706614\\renaissance-als_0"\\als.json" als; \
07:19:16  	if [ $? -eq 0 ] ; then echo ""; echo "renaissance-als_0""_PASSED"; echo ""; cd C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/; rm -f -r "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/\\TKG\\output_16126822706614\\renaissance-als_0"; else echo ""; echo "renaissance-als_0""_FAILED"; echo ""; fi; } 2>&1 | tee -a "C:/Users/jenkins/workspace/Test_openjdk16_hs_extended.perf_x86-64_windows/openjdk-tests/\\TKG\\output_16126822706614\\TestTargetResult";
07:19:21  Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
07:19:21  21/02/07 07:19:20 INFO SparkContext: Running Spark version 2.0.0
07:19:22  21/02/07 07:19:20 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
07:19:22  21/02/07 07:19:21 WARN SparkConf: In Spark 1.0 and later spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone and LOCAL_DIRS in YARN).
07:19:22  21/02/07 07:19:21 INFO SecurityManager: Changing view acls to: jenkins
07:19:22  21/02/07 07:19:21 INFO SecurityManager: Changing modify acls to: jenkins
07:19:22  21/02/07 07:19:21 INFO SecurityManager: Changing view acls groups to: 
07:19:22  21/02/07 07:19:21 INFO SecurityManager: Changing modify acls groups to: 
07:19:22  21/02/07 07:19:21 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(jenkins); groups with view permissions: Set(); users  with modify permissions: Set(jenkins); groups with modify permissions: Set()
07:19:23  21/02/07 07:19:21 INFO PlatformDependent: Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability.
07:19:23  21/02/07 07:19:21 INFO Utils: Successfully started service 'sparkDriver' on port 59540.
07:19:23  21/02/07 07:19:21 INFO SparkEnv: Registering MapOutputTracker
07:19:23  21/02/07 07:19:21 INFO SparkEnv: Registering BlockManagerMaster
07:19:23  21/02/07 07:19:21 INFO DiskBlockManager: Created local directory at C:\Users\jenkins\workspace\Test_openjdk16_hs_extended.perf_x86-64_windows\openjdk-tests\TKG\output_16126822706614\renaissance-als_0\als13081845435272851885\blockmgr-2d63e0fa-990e-403d-833c-0d531dc2ab70
07:19:23  21/02/07 07:19:21 INFO MemoryStore: MemoryStore started with capacity 434.4 MB
07:19:23  21/02/07 07:19:22 INFO SparkEnv: Registering OutputCommitCoordinator
07:19:23  21/02/07 07:19:22 INFO Utils: Successfully started service 'SparkUI' on port 4040.
07:19:23  21/02/07 07:19:22 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.0.0.7:4040
07:19:24  21/02/07 07:19:22 INFO Executor: Starting executor ID driver on host localhost
07:19:24  21/02/07 07:19:23 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 59551.
07:19:24  21/02/07 07:19:23 INFO NettyBlockTransferService: Server created on 10.0.0.7:59551
07:19:24  21/02/07 07:19:23 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 10.0.0.7, 59551)
07:19:24  21/02/07 07:19:23 INFO BlockManagerMasterEndpoint: Registering block manager 10.0.0.7:59551 with 434.4 MB RAM, BlockManagerId(driver, 10.0.0.7, 59551)
07:19:24  21/02/07 07:19:23 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 10.0.0.7, 59551)
07:19:28  Error during tear-down: Cannot invoke "org.apache.spark.mllib.recommendation.MatrixFactorizationModel.userFeatures()" because the return value of "org.renaissance.apache.spark.Als.factModel()" is null
07:19:28  java.lang.NullPointerException: Cannot invoke "org.apache.spark.mllib.recommendation.MatrixFactorizationModel.userFeatures()" because the return value of "org.renaissance.apache.spark.Als.factModel()" is null
07:19:28  	at org.renaissance.apache.spark.Als.tearDownAfterAll(Als.scala:87)
07:19:28  	at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:97)
07:19:28  	at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:308)
07:19:28  	at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:306)
07:19:28  	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
07:19:28  	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
07:19:28  	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
07:19:28  	at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:306)
07:19:28  	at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala)
07:19:28  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
07:19:28  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
07:19:28  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
07:19:28  	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
07:19:28  	at org.renaissance.Launcher.main(Launcher.java:18)
07:19:28  Exception occurred in org.renaissance.apache.spark.Als@5ec1963c: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @5e8ac0e1
07:19:28  java.lang.reflect.InaccessibleObjectException: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @5e8ac0e1
07:19:28  	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357)
07:19:28  	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
07:19:28  	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:177)
07:19:28  	at java.base/java.lang.reflect.Field.setAccessible(Field.java:171)
07:19:28  	at org.apache.spark.util.SizeEstimator$$anonfun$getClassInfo$3.apply(SizeEstimator.scala:336)
07:19:28  	at org.apache.spark.util.SizeEstimator$$anonfun$getClassInfo$3.apply(SizeEstimator.scala:330)
07:19:28  	at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
07:19:28  	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
07:19:28  	at org.apache.spark.util.SizeEstimator$.getClassInfo(SizeEstimator.scala:330)
07:19:28  	at org.apache.spark.util.SizeEstimator$.visitSingleObject(SizeEstimator.scala:222)
07:19:28  	at org.apache.spark.util.SizeEstimator$.org$apache$spark$util$SizeEstimator$$estimate(SizeEstimator.scala:201)
07:19:28  	at org.apache.spark.util.SizeEstimator$.estimate(SizeEstimator.scala:69)
07:19:28  	at org.apache.spark.util.collection.SizeTracker$class.takeSample(SizeTracker.scala:78)
07:19:28  	at org.apache.spark.util.collection.SizeTracker$class.afterUpdate(SizeTracker.scala:70)
07:19:28  	at org.apache.spark.util.collection.SizeTrackingVector.$plus$eq(SizeTrackingVector.scala:31)
07:19:28  	at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:214)
07:19:28  	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:919)
07:19:28  	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:910)
07:19:28  	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:866)
07:19:28  	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:910)
07:19:28  	at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:700)
07:19:28  	at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1213)
07:19:28  	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:103)
07:19:28  	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:86)
07:19:28  	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
07:19:28  	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56)
07:19:28  	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1370)
07:19:28  	at org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:984)
07:19:28  	at org.apache.spark.SparkContext$$anonfun$hadoopFile$1.apply(SparkContext.scala:981)
07:19:28  	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
07:19:28  	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
07:19:28  	at org.apache.spark.SparkContext.withScope(SparkContext.scala:682)
07:19:28  	at org.apache.spark.SparkContext.hadoopFile(SparkContext.scala:981)
07:19:28  	at org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:802)
07:19:28  	at org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:800)
07:19:28  	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
07:19:28  	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
07:19:28  	at org.apache.spark.SparkContext.withScope(SparkContext.scala:682)
07:19:28  	at org.apache.spark.SparkContext.textFile(SparkContext.scala:800)
07:19:28  	at org.renaissance.apache.spark.Als.loadData(Als.scala:59)
07:19:28  	at org.renaissance.apache.spark.Als.setUpBeforeAll(Als.scala:82)
07:19:28  	at org.renaissance.RenaissanceBenchmark.runBenchmark(RenaissanceBenchmark.java:74)
07:19:28  	at org.renaissance.RenaissanceSuite$.$anonfun$main$2(renaissance-suite.scala:308)
07:19:28  	at org.renaissance.RenaissanceSuite$.$anonfun$main$2$adapted(renaissance-suite.scala:306)
07:19:28  	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
07:19:28  	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
07:19:28  	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
07:19:28  	at org.renaissance.RenaissanceSuite$.main(renaissance-suite.scala:306)
07:19:28  	at org.renaissance.RenaissanceSuite.main(renaissance-suite.scala)
07:19:28  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
07:19:28  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
07:19:28  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
07:19:28  	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
07:19:28  	at org.renaissance.Launcher.main(Launcher.java:18)
07:19:28  The following benchmarks failed: als
07:19:28  
07:19:28  renaissance-als_0_FAILED

To rerun: https://ci.adoptopenjdk.net/job/Grinder/parambuild/?JDK_VERSION=16&JDK_IMPL=hotspot&JDK_VENDOR=adoptopenjdk&BUILD_LIST=perf&PLATFORM=x86-64_windows&TARGET=renaissance-als_0

The text was updated successfully, but these errors were encountered:

lumpfish · 2021-02-10T13:07:45Z

Same failure with targets:

renaissance-chi-square_0
renaissance-gauss-mix_0
renaissance-log-regression_0
renaissance-movie-lens_0

JasonFengJ9 · 2021-03-08T22:39:00Z

Can we exclude these failing tests please?

JasonFengJ9 · 2021-03-15T13:30:37Z

@llxia just wondering if the auto exclude applies to this failure or a PR required to be opened manually instead?

llxia · 2021-03-15T13:42:42Z

yes, we can use auto exclude for this.

llxia · 2021-03-15T13:43:14Z

auto exclude test renaissance-als ver=16+

- related: #2253 (comment) Signed-off-by: GitHub <noreply@github.com>

- related: #2253 (comment) Signed-off-by: GitHub <noreply@github.com> Co-authored-by: llxia <llxia@users.noreply.github.com>

lumpfish · 2021-03-15T14:53:06Z

All these benchmarks do not run on jdk16:

renaissance-als_0
renaissance-chi-square_0
renaissance-db-shootout_0
renaissance-finagle-http_0
renaissance-gauss-mix_0
renaissance-log-regression_0
renaissance-mnemonics_0
renaissance-movie-lens_0
renaissance-par-mnemonics_0
renaissance-philosophers_0

smlambert · 2021-03-15T15:06:43Z

Before excluding all, can someone try to put some -add-opens statements on the commandline (which is what the error suggests to do) to see if that is all that is required to have them run successfully? --add-opens java.base/java.util=ALL-UNNAMED etc

smlambert · 2021-03-17T19:13:47Z

These failures are related to https://openjdk.java.net/jeps/396, where --illegal-access option shifted from permit (in jdk9 through jdk15) to deny (in jdk16+).

patkarns · 2021-03-17T21:02:32Z

Before excluding all, can someone try to put some -add-opens statements on the commandline (which is what the error suggests to do) to see if that is all that is required to have them run successfully? --add-opens java.base/java.util=ALL-UNNAMED etc

@smlambert I don't think that's it. After fixing those errors I'm getting this:

03:14:40 java.lang.NullPointerException: Cannot invoke "org.apache.spark.mllib.recommendation.MatrixFactorizationModel.userFeatures()" because the return value of "org.renaissance.apache.spark.Als.factModel()" is null

https://ci.adoptopenjdk.net/job/Grinder/7823/console

smlambert · 2021-03-17T21:09:11Z

@patkarns, yes its fantastic, fails in a new way (so resolved the first issue)... ("Second rule of fight club, once you knock down the first opponent, a new one steps up" - Shelley Lambert)

I ran sanity.perf here: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder_Perf/37/
extended.perf here: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder_Perf/38/

We can report results of test targets affected by this issue when those Grinders complete:
renaissance-als_0 - new/different failure
renaissance-chi-square_0 -
renaissance-db-shootout_0 -
renaissance-finagle-http_0 -
renaissance-gauss-mix_0 -
renaissance-log-regression_0 -
renaissance-mnemonics_0 -
renaissance-movie-lens_0 -
renaissance-par-mnemonics_0 -
renaissance-philosophers_0 -

patkarns · 2021-03-18T00:03:49Z

@patkarns, yes its fantastic, fails in a new way (so resolved the first issue)... ("Second rule of fight club, once you knock down the first opponent, a new one steps up" - Shelley Lambert)

I ran sanity.perf here: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder_Perf/37/
extended.perf here: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder_Perf/38/

We can report results of test targets affected by this issue when those Grinders complete:
renaissance-als_0 - new/different failure
renaissance-chi-square_0 -
renaissance-db-shootout_0 -
renaissance-finagle-http_0 -
renaissance-gauss-mix_0 -
renaissance-log-regression_0 -
renaissance-mnemonics_0 -
renaissance-movie-lens_0 -
renaissance-par-mnemonics_0 -
renaissance-philosophers_0 -

renaissance-als_0 - new/different failure
renaissance-chi-square_0 - java.lang.NullPointerException: Cannot read the array length because "$this" is null
renaissance-db-shootout_0 - java.lang.NullPointerException: Cannot invoke "net.openhft.chronicle.map.ChronicleMap.close()" because "this.map" is null
renaissance-finagle-http_0 - passed
renaissance-gauss-mix_0 - java.lang.NullPointerException: Cannot invoke "org.apache.spark.mllib.clustering.GaussianMixtureModel.gaussians()" because the return value of "org.renaissance.apache.spark.GaussMix.gmm()" is null
renaissance-log-regression_0 - java.lang.NullPointerException: Cannot invoke "org.apache.spark.ml.classification.LogisticRegressionModel.coefficients()" because the return value of "org.renaissance.apache.spark.LogRegression.mlModel()" is null
renaissance-mnemonics_0 - passed
renaissance-movie-lens_0 - Exception occurred in org.renaissance.apache.spark.MovieLens@c554aebd: null java.lang.ExceptionInInitializerError
renaissance-par-mnemonics_0 - passed
renaissance-philosophers_0 - passed

smlambert · 2021-04-13T19:08:56Z

Close in favour of #2500

lumpfish mentioned this issue Feb 10, 2021

JDK16 Triage push Coordination #2252

Closed

karianna added the bug label Feb 12, 2021

smlambert self-assigned this Feb 19, 2021

This was referenced Feb 23, 2021

Update Renaissance suite to latest release v0.11.0 #2298

Closed

renaissance-dec-tree_0_FAILED - Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 0 eclipse-openj9/openj9#12078

Closed

github-actions bot pushed a commit that referenced this issue Mar 15, 2021

AUTO: auto exclude test renaissance-als ver=16+

e18ca1e

- related: #2253 (comment) Signed-off-by: GitHub <noreply@github.com>

github-actions bot mentioned this issue Mar 15, 2021

auto exclude test renaissance-als ver=16+ #2379

Merged

github-actions bot added the test excluded label Mar 15, 2021

smlambert pushed a commit that referenced this issue Mar 15, 2021

AUTO: auto exclude test renaissance-als ver=16+ (#2379)

013e76c

- related: #2253 (comment) Signed-off-by: GitHub <noreply@github.com> Co-authored-by: llxia <llxia@users.noreply.github.com>

patkarns self-assigned this Mar 17, 2021

smlambert mentioned this issue Mar 25, 2021

Add --add-opens for java.lang + java.io in system_custom by default #2413

Merged

andrew-m-leonard mentioned this issue Apr 6, 2021

openj9-0.26.0-m2 triage adoptium/temurin-build#2556

Closed

patkarns mentioned this issue Apr 7, 2021

Renaissance als failure #2468

Merged

smlambert mentioned this issue Apr 13, 2021

extended.perf: several renaissance tests fails across all platforms and VMs on JDK16+ #2500

Closed

smlambert closed this as completed Apr 13, 2021

karianna added this to the April 2021 milestone Apr 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

jdk16 extended.perf renaissance-als_0 java.lang.reflect.InaccessibleObjectException #2253

jdk16 extended.perf renaissance-als_0 java.lang.reflect.InaccessibleObjectException #2253

lumpfish commented Feb 10, 2021 •

edited

Loading

lumpfish commented Feb 10, 2021

JasonFengJ9 commented Mar 8, 2021

JasonFengJ9 commented Mar 15, 2021

llxia commented Mar 15, 2021

llxia commented Mar 15, 2021

lumpfish commented Mar 15, 2021

smlambert commented Mar 15, 2021

smlambert commented Mar 17, 2021

patkarns commented Mar 17, 2021

smlambert commented Mar 17, 2021

patkarns commented Mar 18, 2021 •

edited

Loading

smlambert commented Apr 13, 2021

jdk16 extended.perf renaissance-als_0 java.lang.reflect.InaccessibleObjectException #2253

jdk16 extended.perf renaissance-als_0 java.lang.reflect.InaccessibleObjectException #2253

Comments

lumpfish commented Feb 10, 2021 • edited Loading

lumpfish commented Feb 10, 2021

JasonFengJ9 commented Mar 8, 2021

JasonFengJ9 commented Mar 15, 2021

llxia commented Mar 15, 2021

llxia commented Mar 15, 2021

lumpfish commented Mar 15, 2021

smlambert commented Mar 15, 2021

smlambert commented Mar 17, 2021

patkarns commented Mar 17, 2021

smlambert commented Mar 17, 2021

patkarns commented Mar 18, 2021 • edited Loading

smlambert commented Apr 13, 2021

lumpfish commented Feb 10, 2021 •

edited

Loading

patkarns commented Mar 18, 2021 •

edited

Loading