Skip to content

Commit 1e15e3f

Browse files
[SPARK-48843] Prevent infinite loop with BindParameters
### What changes were proposed in this pull request? In order to resolve the named parameters on the subtree, BindParameters recurses into the subtrees and tries to match the pattern with the named parameters. If there's no named parameter in the current level, the rule tries to return the unchanged plan. However, instead of returning the current plan object, the rule always returns the captured root plan node, leading into the infinite recursion. ### Why are the changes needed? Infinite recursion with the named parameters and the global limit. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added unit tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #47271 from nemanja-boric-databricks/fix-bind. Lead-authored-by: Nemanja Boric <nemanja.boric@databricks.com> Co-authored-by: Wenchen Fan <cloud0fan@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit a39f70d) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 67047cd commit 1e15e3f

File tree

2 files changed

+8
-1
lines changed

2 files changed

+8
-1
lines changed

python/pyspark/sql/tests/connect/test_connect_basic.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1242,6 +1242,13 @@ def test_sql_with_named_args(self):
12421242
df2 = self.spark.sql("SELECT * FROM range(10) WHERE id > :minId", args={"minId": 7})
12431243
self.assert_eq(df.toPandas(), df2.toPandas())
12441244

1245+
def test_namedargs_with_global_limit(self):
1246+
sqlText = """SELECT * FROM VALUES (TIMESTAMP('2022-12-25 10:30:00'), 1) as tab(date, val)
1247+
where val = :val"""
1248+
df = self.connect.sql(sqlText, args={"val": 1})
1249+
df2 = self.spark.sql(sqlText, args={"val": 1})
1250+
self.assert_eq(df.toPandas(), df2.toPandas())
1251+
12451252
def test_sql_with_pos_args(self):
12461253
df = self.connect.sql("SELECT * FROM range(10) WHERE id > ?", args=[7])
12471254
df2 = self.spark.sql("SELECT * FROM range(10) WHERE id > ?", args=[7])

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/parameters.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ object BindParameters extends Rule[LogicalPlan] with QueryErrorsBase {
136136
args(posToIndex(pos))
137137
}
138138

139-
case _ => plan
139+
case other => other
140140
}
141141
}
142142
}

0 commit comments

Comments
 (0)