feat: Support Struct field selection in the SQL engine, RENAME and REPLACE select wildcard options

[alexander-beedie]

Closes #16651.

• Adds SQL support for Struct field selection and wildcard expansion.
• Adds support for SELECT * RENAME (…) syntax (also works with struct wildcards).
• Adds support for SELECT * REPLACE (…) syntax (also works with struct wildcards).

Also

• Fixed issue where ORDER BY <n> ordinal syntax picked the wrong columns to sort.
• Consolidated QualifiedWildcard & CompoundIdentifier SQL → Expr translation.
• Generate informative error on use of currently unsupported wildcard options, eg:
  SELECT tbl.* ILIKE … → "ILIKE wildcard option is currently unsupported".

Note

• Supporting use of Struct fields in JOIN constraints requires some more involved updates to table/column resolution, so that will come as a separate/follow-up PR (this one is complicated enough already!) 🤔
• We will be able to use both REPLACE and RENAME modifiers in the same statement once this upstream fix (Fix Snowflake SELECT * wildcard REPLACE ... RENAME order apache/datafusion-sqlparser-rs#1321) is available.

---

Example

import polars as pl

df = pl.DataFrame(
  {
    "id": [100, 200, 300, 400],
    "name": ["Alice", "Bob", "David", "Zoe"],
    "age": [32, 27, 19, 45],
    "other": [{"n": 1.5}, {"n": None}, {"n": -0.5}, {"n": 2.0}],
  }
).select(pl.struct(pl.all()).alias("json_msg"))

# ┌─────────────────────────┐
# │ json_msg                │
# │ ---                     │
# │ struct[4]               │
# ╞═════════════════════════╡
# │ {100,"Alice",32,{1.5}}  │
# │ {200,"Bob",27,{null}}   │
# │ {300,"David",19,{-0.5}} │
# │ {400,"Zoe",45,{2.0}}    │
# └─────────────────────────┘

Select, filter, and sort on Struct field values from SQL:

df.sql(
  """
  SELECT
    json_msg.id   AS ID,
    json_msg.name AS NAME,
    json_msg.age  AS AGE
  FROM
    self
  WHERE
    json_msg.age > 20 AND
    json_msg.other.n IS NOT NULL
  ORDER BY
    json_msg.name DESC
  """
)
# ┌─────┬───────┬─────┐
# │ ID  ┆ NAME  ┆ AGE │
# │ --- ┆ ---   ┆ --- │
# │ i64 ┆ str   ┆ i64 │
# ╞═════╪═══════╪═════╡
# │ 400 ┆ Zoe   ┆ 45  │
# │ 100 ┆ Alice ┆ 32  │
# └─────┴───────┴─────┘

Wildcard Struct selection/expansion, with REPLACE wildcard option:

df.sql(
  """
  SELECT json_msg.* REPLACE (UPPER(name) AS name) FROM self
  """
)
# shape: (4, 4)
# ┌─────┬───────┬─────┬───────────┐
# │ id  ┆ name  ┆ age ┆ other     │
# │ --- ┆ ---   ┆ --- ┆ ---       │
# │ i64 ┆ str   ┆ i64 ┆ struct[1] │
# ╞═════╪═══════╪═════╪═══════════╡
# │ 100 ┆ ALICE ┆ 32  ┆ {1.5}     │
# │ 200 ┆ BOB   ┆ 27  ┆ {null}    │
# │ 300 ┆ DAVID ┆ 19  ┆ {-0.5}    │
# │ 400 ┆ ZOE   ┆ 45  ┆ {2.0}     │
# └─────┴───────┴─────┴───────────┘

Wildcard selection/expansion of nested Struct, with RENAME wildcard option:

df.sql(
  """
  SELECT json_msg.other.* RENAME (n AS value) FROM self
  """
)
# shape: (4, 1)
# ┌───────┐
# │ value │
# │ ---   │
# │ f64   │
# ╞═══════╡
# │ 1.5   │
# │ null  │
# │ -0.5  │
# │ 2.0   │
# └───────┘

[codecov]

Codecov Report

Attention: Patch coverage is 94.76190% with 11 lines in your changes missing coverage. Please review.

Project coverage is 80.89%. Comparing base (8a6bf4b) to head (64982d7).
Report is 27 commits behind head on main.

Files	Patch %	Lines
crates/polars-sql/src/sql_expr.rs	87.09%	8 Missing ⚠️
crates/polars-sql/src/context.rs	97.94%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #17109      +/-   ##
==========================================
+ Coverage   80.86%   80.89%   +0.03%     
==========================================
  Files        1456     1456              
  Lines      191141   191403     +262     
  Branches     2728     2742      +14     
==========================================
+ Hits       154562   154836     +274     
+ Misses      36073    36058      -15     
- Partials      506      509       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.