Skip to content

Commit 271c891

Browse files
rednaxelafxcloud-fan
authored andcommitted
[SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars as transient
## What changes were proposed in this pull request? Mark `HashAggregateExec.bufVars` as transient to avoid it from being serialized. Also manually null out this field at the end of `doProduceWithoutKeys()` to shorten its lifecycle, because it'll no longer be used after that. ## How was this patch tested? Existing tests. Author: Kris Mok <kris.mok@databricks.com> Closes #21039 from rednaxelafx/codegen-improve.
1 parent c604d65 commit 271c891

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,8 +174,8 @@ case class HashAggregateExec(
174174
}
175175
}
176176

177-
// The variables used as aggregation buffer. Only used for aggregation without keys.
178-
private var bufVars: Seq[ExprCode] = _
177+
// The variables used as aggregation buffer. Only used in codegen for aggregation without keys.
178+
@transient private var bufVars: Seq[ExprCode] = _
179179

180180
private def doProduceWithoutKeys(ctx: CodegenContext): String = {
181181
val initAgg = ctx.addMutableState(CodeGenerator.JAVA_BOOLEAN, "initAgg")
@@ -238,6 +238,8 @@ case class HashAggregateExec(
238238
| }
239239
""".stripMargin)
240240

241+
bufVars = null // explicitly null this field out to allow the referent to be GC'd sooner
242+
241243
val numOutput = metricTerm(ctx, "numOutputRows")
242244
val aggTime = metricTerm(ctx, "aggTime")
243245
val beforeAgg = ctx.freshName("beforeAgg")

0 commit comments

Comments
 (0)