Skip to content

Conversation

@simonvandel
Copy link
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

I noticed in #17278 that from without an explicit projection did not work.
This should fix it.

The from-first syntax is also available in duckdb. See https://duckdb.org/docs/stable/sql/query_syntax/from.html#examples

What changes are included in this PR?

Are these changes tested?

Yes, added SLT test

Are there any user-facing changes?

FROM-first syntax without projection is now supported.

I noticed in apache#17278 that `from`
without an explicit projection did not work.
This should fix it
@github-actions github-actions bot added sql SQL Planner sqllogictest SQL Logic Tests (.slt) labels Aug 22, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯 -- thank you @simonvandel

I also verified duckdb does support this

D create table t(x int);
D insert into t values (1);
D from t;
┌───────┐
│   x   │
│ int32 │
├───────┤
│   1   │
└───────┘

@Jefffrey or @jonahgao -- any concern about supporting this syntax?

@alamb alamb changed the title Support from-first syntax Support from-first SQL syntax Aug 23, 2025
Copy link
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used this in DuckDB before and agree it's pretty slick & handy;

Maybe need to update the SQL reference docs with this new behaviour?

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Aug 23, 2025
@simonvandel
Copy link
Contributor Author

Maybe need to update the SQL reference docs with this new behaviour?

* https://datafusion.apache.org/user-guide/sql/select.html

I added some documentation in 9f1b436

Copy link
Contributor

@Jefffrey Jefffrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@Jefffrey Jefffrey merged commit 85f879d into apache:main Aug 24, 2025
28 checks passed
adriangb added a commit to pydantic/datafusion that referenced this pull request Sep 11, 2025
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.

The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection

Also updates the test expectation to correctly show the empty Projection
node in the query plan.

Fixes apache#17513
@alamb
Copy link
Contributor

alamb commented Sep 11, 2025

@xudong963 found this PR seems to have introduced a regression

@adriangb has a proposed PR up to fix:

@adriangb
Copy link
Contributor

@alamb we discussed a bit in #17520 (comment), I'm not sure that this PR introduced a bug. If I understand correctly select from t1 should be equivalent to select * from t1 -> then there is no bug. Now if select from t1 is meant to be select every row but no columns then there is a bug but it has nothing to do with the where clause or projection pushdown.

FWIW:

DataFusion CLI v50.0.0
> create table t1 (a int, b int);
0 row(s) fetched. 
Elapsed 0.004 seconds.

> insert into t1 values (1, 2);
+-------+
| count |
+-------+
| 1     |
+-------+
1 row(s) fetched. 
Elapsed 0.010 seconds.

> select * from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. 
Elapsed 0.003 seconds.

> select from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. 
Elapsed 0.004 seconds.

> from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. 
Elapsed 0.001 seconds.

Postgres:

postgres=# create table t1 (a int, b int);
CREATE TABLE
postgres=# insert into t1 values (1), (2);
INSERT 0 2
postgres=# select * from t1;
 a | b 
---+---
 1 |  
 2 |  
(2 rows)

postgres=# select from t1;
--
(2 rows)

postgres=# from t1;
ERROR:  syntax error at or near "from"
LINE 1: from t1;

DuckDB:

DuckDB v1.3.2 (Ossivalis) 0b83e5d2f6
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D create table t1 AS (select 1 as a, 2 as b);
D insert into t1 values (1, 2);
D select from t1;
Parser Error:
SELECT clause without selection list
D from t1;
┌───────┬───────┐
│   a   │   b   │
│ int32 │ int32 │
├───────┼───────┤
│     12 │
│     12 │
└───────┴───────┘

@adriangb
Copy link
Contributor

adriangb commented Sep 11, 2025

So maybe the answer is that we should make select from t1; equivalent to "select no columns from t1" while keeping from t1; equivalent to select * from t1;?

@alamb
Copy link
Contributor

alamb commented Sep 11, 2025

So maybe the answer is that we should make select from t1; equivalent to "select no columns from t1" while keeping from t1; equivalent to select * from t1;?

I think we should be consistent with other databases

This proposal seems reasonable to me

SHould we also make from t (without anything else) an error like it is in duckdb?

@adriangb
Copy link
Contributor

SHould we also make from t (without anything else) an error like it is in duckdb?

DuckDB does support FROM t;, it does not support SELECT FROM t;

alamb pushed a commit that referenced this pull request Sep 12, 2025
* Add failing test

* Fix regression in SELECT FROM syntax with WHERE clause

When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR #17295 which added FROM-first syntax support.

The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection

Also updates the test expectation to correctly show the empty Projection
node in the query plan.

Fixes #17513

* Revert

* Fix regression: SELECT FROM syntax should return empty projection

Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.

Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.

Fixes #17513

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* add a better regression test

* remove comment

* fmt

* Update datafusion/sqllogictest/test_files/projection.slt

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* Update datafusion/core/tests/sql/select.rs

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* revert docs

* fmt

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
alamb pushed a commit to alamb/datafusion that referenced this pull request Sep 12, 2025
* Add failing test

* Fix regression in SELECT FROM syntax with WHERE clause

When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.

The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection

Also updates the test expectation to correctly show the empty Projection
node in the query plan.

Fixes apache#17513

* Revert

* Fix regression: SELECT FROM syntax should return empty projection

Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.

Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.

Fixes apache#17513

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* add a better regression test

* remove comment

* fmt

* Update datafusion/sqllogictest/test_files/projection.slt

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* Update datafusion/core/tests/sql/select.rs

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* revert docs

* fmt

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
xudong963 pushed a commit that referenced this pull request Sep 13, 2025
* Add failing test

* Fix regression in SELECT FROM syntax with WHERE clause

When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR #17295 which added FROM-first syntax support.

The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection

Also updates the test expectation to correctly show the empty Projection
node in the query plan.

Fixes #17513

* Revert

* Fix regression: SELECT FROM syntax should return empty projection

Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.

Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.

Fixes #17513

🤖 Generated with [Claude Code](https://claude.ai/code)



* add a better regression test

* remove comment

* fmt

* Update datafusion/sqllogictest/test_files/projection.slt



* Update datafusion/core/tests/sql/select.rs



* revert docs

* fmt

---------

Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
samueleresca pushed a commit to samueleresca/datafusion that referenced this pull request Sep 15, 2025
* Add failing test

* Fix regression in SELECT FROM syntax with WHERE clause

When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.

The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection

Also updates the test expectation to correctly show the empty Projection
node in the query plan.

Fixes apache#17513

* Revert

* Fix regression: SELECT FROM syntax should return empty projection

Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.

Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.

Fixes apache#17513

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* add a better regression test

* remove comment

* fmt

* Update datafusion/sqllogictest/test_files/projection.slt

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* Update datafusion/core/tests/sql/select.rs

Co-authored-by: Oleks V <comphead@users.noreply.github.com>

* revert docs

* fmt

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants