Create Spark builtin function array_contains #20611
Description
Is your feature request related to a problem or challenge?
Now Comet uses array_has for Apache Spark array_contains. However the Spark counterpart has specifics working with NULLS, namely
scala> spark.sql("select array_contains(array(1, null, 3), 2)").show(false)
+------------------------------------+
|array_contains(array(1, NULL, 3), 2)|
+------------------------------------+
|null |
+------------------------------------+
Spark behavior explained by array element comparison for NULLs when comparing NULL with scalar returns NULL. More on this https://issues.apache.org/jira/browse/SPARK-55749
The same time DataFusion/DuckDB returns false.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response