Skip to content

Return an error when a column does not exist in window function#9202

Merged
alamb merged 9 commits into
apache:mainfrom
PhVHoang:phvhoang/fix-panic
Feb 14, 2024
Merged

Return an error when a column does not exist in window function#9202
alamb merged 9 commits into
apache:mainfrom
PhVHoang:phvhoang/fix-panic

Conversation

@PhVHoang

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #9166.

Rationale for this change

To return an error instead of panicking when columns don't exist in window function.

What changes are included in this PR?

Code, tests

Are these changes tested?

Yes

Are there any user-facing changes?

Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @PhVHoang -- this is a great first contribution.

I think the code can be made simpler and I left a comment for your consideration

Comment thread datafusion/sql/src/expr/function.rs Outdated
let idx_result = schema.index_of_column(col);

// Propagate the error to the caller and return it immediately
let idx = idx_result

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the ok() function call here discards the error.

I think changing

                        let idx = schema.index_of_column(col).unwrap();

To

                        let idx = schema.index_of_column(col).ok()?;

accomplishes the same thing (and your test still passes)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your suggestion. Yes it's way simpler than that of mine.

Fixed.

Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @PhVHoang -- this looks good to me

I think it would be good if @mustafasrepo could also review this PR before we merge, if he has time

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @PhVHoang
its better than crash

@comphead

Copy link
Copy Markdown
Contributor

@PhVHoang please add the test also for non existent partition by column
and qualified column like t.a, schema.t.a, catalog.schema.t.a

@mustafasrepo mustafasrepo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @PhVHoang for this PR. LGTM!

@alamb

alamb commented Feb 14, 2024

Copy link
Copy Markdown
Contributor

@PhVHoang please add the test also for non existent partition by column and qualified column like t.a, schema.t.a, catalog.schema.t.a

I think the only thing remaining before merging this PR is the suggestion from @comphead #9202 (comment) for a few more tests. I can add them hopefully later today if no one else has a chance

Signed-off-by: Hoang Pham <a2k40pbc@gmail.com>
@PhVHoang

Copy link
Copy Markdown
Contributor Author

@alamb @comphead I added some more tests. Please take a look and let me know if these tests are sufficient for this change.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @PhVHoang

@alamb
alamb merged commit 0444dad into apache:main Feb 14, 2024
@alamb

alamb commented Feb 14, 2024

Copy link
Copy Markdown
Contributor

Thanks again everyone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid column reference in window function order by causes panic

4 participants