Skip to content

Commit cbd891d

Browse files
committed
update3
1 parent bc1b7e5 commit cbd891d

File tree

1 file changed

+9
-7
lines changed
  • sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate

1 file changed

+9
-7
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -400,13 +400,15 @@ abstract class DeclarativeAggregate
400400
final override def aggBufferSchema: StructType = StructType.fromAttributes(aggBufferAttributes)
401401

402402
final lazy val inputAggBufferAttributes: Seq[AttributeReference] = {
403-
// SPARK-31620: inputAggBufferAttributes from a partial agg can be referenced by a final agg
404-
// in order to merge agg values. However, in case of an aggregate function contains a subquery,
405-
// the aggregate function will be transformed to a new copied node during `PlanSubqueries` and
406-
// lost original attributes because `TreeNode` does not preserve "lazy val" during `makeCopy`.
407-
// As a result, the final agg could fail to resolve references through partial agg. So we use
408-
// the tag to save the original attributes to let the new copied node share the same attributes
409-
// with old node.
403+
// SPARK-31620: inputAggBufferAttributes from a partial aggregate can be referenced by a final
404+
// aggregate in order to merge aggregate values. However, in case of an aggregate function
405+
// contains a subquery, the aggregate function will be transformed to a new copied node during
406+
// `PlanSubqueries` and lost original attributes because `TreeNode` does not preserve them
407+
// during `makeCopy`. As a result, the final aggregate could fail to resolve references through
408+
// partial aggregate's output. So, we use the tag to save the original attributes to let the
409+
// new copied node share the same attributes with old node. Note, we don't save other attributes
410+
// within an aggregate function and ImperativeAggregate's inputAggBufferAttributes because they
411+
// will not be referenced out of the aggregate function itself.
410412
getTagValue(inputAggBufferAttributeTag).getOrElse {
411413
val attrs = aggBufferAttributes.map(_.newInstance())
412414
setTagValue(inputAggBufferAttributeTag, attrs)

0 commit comments

Comments
 (0)