Skip to content

Commit cd870c0

Browse files
committed
[SPARK-20940][CORE] Replace IllegalAccessError with IllegalStateException
## What changes were proposed in this pull request? `IllegalAccessError` is a fatal error (a subclass of LinkageError) and its meaning is `Thrown if an application attempts to access or modify a field, or to call a method that it does not have access to`. Throwing a fatal error for AccumulatorV2 is not necessary and is pretty bad because it usually will just kill executors or SparkContext ([SPARK-20666](https://issues.apache.org/jira/browse/SPARK-20666) is an example of killing SparkContext due to `IllegalAccessError`). I think the correct type of exception in AccumulatorV2 should be `IllegalStateException`. ## How was this patch tested? Jenkins Author: Shixiong Zhu <shixiong@databricks.com> Closes #18168 from zsxwing/SPARK-20940. (cherry picked from commit 24db358) Signed-off-by: Shixiong Zhu <shixiong@databricks.com>
1 parent 9846a3c commit cd870c0

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

core/src/main/scala/org/apache/spark/util/AccumulatorV2.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ abstract class AccumulatorV2[IN, OUT] extends Serializable {
6767

6868
private def assertMetadataNotNull(): Unit = {
6969
if (metadata == null) {
70-
throw new IllegalAccessError("The metadata of this accumulator has not been assigned yet.")
70+
throw new IllegalStateException("The metadata of this accumulator has not been assigned yet.")
7171
}
7272
}
7373

@@ -249,7 +249,7 @@ private[spark] object AccumulatorContext {
249249
// Since we are storing weak references, we must check whether the underlying data is valid.
250250
val acc = ref.get
251251
if (acc eq null) {
252-
throw new IllegalAccessError(s"Attempted to access garbage collected accumulator $id")
252+
throw new IllegalStateException(s"Attempted to access garbage collected accumulator $id")
253253
}
254254
acc
255255
}

core/src/test/scala/org/apache/spark/AccumulatorSuite.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -208,7 +208,7 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with LocalSparkContex
208208
assert(ref.get.isEmpty)
209209

210210
// Getting a garbage collected accum should throw error
211-
intercept[IllegalAccessError] {
211+
intercept[IllegalStateException] {
212212
AccumulatorContext.get(accId)
213213
}
214214

0 commit comments

Comments
 (0)