docs: Add `WITH ORDINALITY` #33317

ggevay · 2025-08-13T15:35:28Z

As it says on the tin.

Our support for WITH ORDINALITY used to have serious issues, so it was better that it was not mentioned in our docs up to now. But we've fixed these issues recently (#33163, #33291, #33272), so it's time to add it to the docs.

Motivation

Tips for reviewer

Checklist

This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

kay-kim

lgtm -- left a suggestion.

kay-kim · 2025-08-13T16:01:16Z

doc/user/data/sql_funcs.yml

+    Table functions evaluate to a set of rows, rather than a single expression.
+    You can also add `WITH ORDINALITY` after the closing parenthesis of a table function call in a `FROM` clause.
+    This causes an `ordinality` column to appear, which is a 1-based numbering of the output rows of each call of
+    the table function. For example, `unnest(list) WITH ORDINALITY` numbers the unnested elements of each input list.


Maybe?

Suggested change

Table functions evaluate to a set of rows, rather than a single expression.

You can also add `WITH ORDINALITY` after the closing parenthesis of a table function call in a `FROM` clause.

This causes an `ordinality` column to appear, which is a 1-based numbering of the output rows of each call of

the table function. For example, `unnest(list) WITH ORDINALITY` numbers the unnested elements of each input list.

Table functions evaluate to a set of rows rather than a single expression.

{{< annotation type="WITH ORDINALITY" >}}

You can also add `WITH ORDINALITY` after the table function call in a `FROM`

clause. This causes an `ordinality` column to appear, which is a 1-based

numbering of the output rows of each call of the table function.

For example, `SELECT * FROM unnest(LIST[5,10]) WITH ORDINALITY AS

list_item;` (or `SELECT * FROM unnest(LIST[5,10]) AS list_item WITH

ORDINALITY;`) includes the `ordinality` column containing the 1-based

numbering of the unnested elements.

{{</ annotation >}}

Thank you! I've made some further adjustments:

"after the table function call" -> "after a table function call", because there can be multiple table function calls in a FROM clause.

I've dropped the "(or SELECT * FROM unnest(LIST[5,10]) AS list_item WITH ORDINALITY;)", because this is not a syntax that I'd like people to use. I think no other system allows this. We used to allow only this syntax, but that was just a parser glitch.

"of the unnested elements" -> "of the unnested items", to align the wording with the above list_item column name.

Works for me!

ggevay · 2025-08-13T16:25:51Z

Thank you for the review!

ggevay · 2025-08-13T16:26:13Z

Not merging yet. I'll merge it, when the fixes go out next week Aug 21.

ggevay · 2025-08-13T18:01:52Z

I've just realized that ROWS FROM was also undocumented. I've added one more commit to the PR to document this as well. Could you check it @kay-kim ? I'm not sure whether this {{< annotation type="ROWS FROM" >}} box is appropriate for this one as well, because this one is bigger than the other box on WITH ORDINALITY.

kay-kim · 2025-08-13T18:41:27Z

doc/user/data/sql_funcs.yml

+
+    {{</ annotation >}}
+
+    {{< annotation type="ROWS FROM" >}}


Oh ... I wonder if most of the content should be actually an idiomatic SQL thing for ROWS FROM and we just link to that from here instead of a whole annotation thing.

Well, the idiomatic SQL pages are for when there is a particular preferred way of expressing a thing in Materialize, and it is possibly different from other systems' preferred way. ROWS FROM is not like that: you can simply write it the same way in Materialize as in other systems.

Postgres documents both WITH ORDINALITY and ROWS FROM in the SELECT page, at the FROM clause (at function_name):
https://www.postgresql.org/docs/current/sql-select.html#SQL-FROM
We could do the same, i.e., expand table_expr in our SELECT page.

Ah, got it. In that case, 100% agree with your suggestion to move these to our select page and just link to there.

Ok, I'll do that move now.

Actually, how about we create a separate page just for table function calls? The problem is that both table_expr/table_ref in the SELECT page and table_ref in the JOIN page would need to allow WITH ORDINALITY and ROWS FROM, but we don't want to repeat this on both pages. So, we could create a separate page, and just link to there from the SELECT page and JOIN page (and table function list) where we explain table_expr. (table_ref should be table_expr everywhere.)

Having a separate page would be similar also to how the various extra things around aggregation function calls are explained in separate pages, e.g.

The FILTER clause: https://materialize.com/docs/sql/functions/filters/

ORDER BY inside an aggregation: https://materialize.com/docs/sql/functions/list_agg/

That works. Redoing the SELECT/JOIN/CTE pages are on my todo list; so, keeping this change clean works for me.

Could you please check again @kay-kim ?

I've pushed a commit for the above:

Created a new table functions page, moved the WITH ORDINALITY and ROWS FROM content to it, plus added some more content, and linked it in a few places.

fixed some minor things:

The SELECT page had table_ref and table_expr inconsistently. Now it has table_expr in both places. (It would have been good to also modify the same table_ref to table_expr in the JOIN page, but I didn't do that, because I would have needed to touch the grammar, but sql-grammar/generate.sh was not really working for me.)

I've fixed the most pressing part of https://github.com/MaterializeInc/database-issues/issues/8311#issuecomment-3201977113

Thank you 😍 😍😍 ... Added a patch with some minor tweaks.

ggevay · 2025-08-20T07:27:42Z

Thank you!

I'll merge it on Thursday after the release is rolled out.

ggevay requested a review from a team as a code owner August 13, 2025 15:35

kay-kim approved these changes Aug 13, 2025

View reviewed changes

ggevay force-pushed the with-ordinality-docs branch from 51f0fb1 to 7f405e8 Compare August 13, 2025 16:07

ggevay force-pushed the with-ordinality-docs branch from 762e175 to 7f7c9df Compare August 13, 2025 18:08

kay-kim reviewed Aug 13, 2025

View reviewed changes

ggevay added 2 commits August 19, 2025 13:19

docs: Add WITH ORDINALITY

2ffa74d

docs: add ROWS FROM

44f2cdc

ggevay force-pushed the with-ordinality-docs branch 2 times, most recently from 2656e5c to 8e85e6a Compare August 19, 2025 19:55

docs: Create a page for table functions + other minor improvements

2c82dae

ggevay force-pushed the with-ordinality-docs branch from 8e85e6a to 2c82dae Compare August 19, 2025 19:57

tweaks

0dcf8cf

ggevay force-pushed the with-ordinality-docs branch from 2b1557f to 0dcf8cf Compare August 20, 2025 07:26

ggevay merged commit 096b3ec into MaterializeInc:main Aug 21, 2025
9 checks passed

-    Table functions evaluate to a set of rows, rather than a single expression.
-    You can also add `WITH ORDINALITY` after the closing parenthesis of a table function call in a `FROM` clause.
-    This causes an `ordinality` column to appear, which is a 1-based numbering of the output rows of each call of
-    the table function. For example, `unnest(list) WITH ORDINALITY` numbers the unnested elements of each input list.
+    Table functions evaluate to a set of rows rather than a single expression.
+    {{< annotation type="WITH ORDINALITY" >}}
+    You can also add `WITH ORDINALITY` after the table function call in a `FROM`
+    clause. This causes an `ordinality` column to appear, which is a 1-based
+    numbering of the output rows of each call of the table function.
+    For example, `SELECT * FROM unnest(LIST[5,10]) WITH ORDINALITY AS
+    list_item;` (or `SELECT * FROM unnest(LIST[5,10]) AS list_item WITH
+    ORDINALITY;`) includes the `ordinality` column containing the 1-based
+    numbering of the unnested elements.
+    {{</ annotation >}}

docs: Add WITH ORDINALITY #33317

docs: Add WITH ORDINALITY #33317

Uh oh!

Conversation

ggevay commented Aug 13, 2025

Motivation

Tips for reviewer

Checklist

Uh oh!

kay-kim left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ggevay commented Aug 13, 2025

Uh oh!

ggevay commented Aug 13, 2025

Uh oh!

ggevay commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ggevay Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ggevay commented Aug 20, 2025

Uh oh!

Uh oh!

Uh oh!

docs: Add `WITH ORDINALITY` #33317

docs: Add `WITH ORDINALITY` #33317

ggevay commented Aug 13, 2025 •

edited

Loading

ggevay Aug 19, 2025 •

edited

Loading