Skip to content

Conversation

@irenjj
Copy link
Contributor

@irenjj irenjj commented Feb 28, 2025

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@irenjj irenjj marked this pull request as ready for review February 28, 2025 14:32
@github-actions github-actions bot added the core Core DataFusion crate label Feb 28, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @irenjj -- I am not 100% sure about this implementation; let me know what you think

cc @milenkovicm

expr_planners: Vec<Arc<dyn ExprPlanner>>,
) -> Self {
self.expr_planners = Some(expr_planners);
if self.expr_planners.is_none() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By putting these checks here I think the API will be quite confusing to use -- it means that calling with_expr_planners and the other methods below will only set the fields if there is no existing field set and silently ignore the argument if there is already planners specified

/// scalar, aggregate and windows functions.
/// For each setter method call, default values will only be created and set if the corresponding
/// field is currently None, otherwise the existing value will be preserved.
pub fn with_default_features(self) -> Self {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @irenjj

I thought the idea was that with_default_features would be additive to the existing set of features

That is, instead of replacing any existing factors/formats/etc this methods would append the default features

So instead of

        self.with_table_factories(SessionStateDefaults::default_table_factories())

Something more like

let mut table_factories = self.with_table_factories.take().unwrap_or_default();
table_factories.extend(SessionStateDefaults::default_table_factories())
self.table_factories = Some(table_factories)

🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. I think the second approach is more suitable.

@milenkovicm
Copy link
Contributor

I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @irenjj -- I think this makes a lot of sense to me

I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?

@milenkovicm I think the usecase of not automatically registering all the default features is to allow creating a SessionState without the defaults (e.g. if you don't want to defaults).

Since there is no method like SessionStateBuilder::clear I am not sure how else one would accomplish this usecase 🤔

@milenkovicm
Copy link
Contributor

Thank you @irenjj -- I think this makes a lot of sense to me

I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?

@milenkovicm I think the usecase of not automatically registering all the default features is to allow creating a SessionState without the defaults (e.g. if you don't want to defaults).

Since there is no method like SessionStateBuilder::clear I am not sure how else one would accomplish this usecase 🤔

than makes sense, what I suggest is to keep all (new) methods as they are just provide new method which will create new with defaults. but I don't have strong opinion about it

@alamb alamb changed the title chore: forbide with_default_features override existing information chore: Update SessionStateBuilder::with_default_features does not replace existing features Mar 3, 2025
@alamb
Copy link
Contributor

alamb commented Mar 3, 2025

I also made a small PR to try and clarify the docs:

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@alamb
Copy link
Contributor

alamb commented Mar 3, 2025

Thank you @irenjj -- I think this makes a lot of sense to me

I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?

@milenkovicm I think the usecase of not automatically registering all the default features is to allow creating a SessionState without the defaults (e.g. if you don't want to defaults).
Since there is no method like SessionStateBuilder::clear I am not sure how else one would accomplish this usecase 🤔

than makes sense, what I suggest is to keep all (new) methods as they are just provide new method which will create new with defaults. but I don't have strong opinion about it

@alamb alamb merged commit b377725 into apache:main Mar 4, 2025
24 checks passed
@alamb
Copy link
Contributor

alamb commented Mar 4, 2025

Thanks again @irenjj and @milenkovicm

It seems as if @shruti2522 already has a PR to implement the new_with_default_features 🎉

@irenjj irenjj deleted the with_default_features_not_override branch March 4, 2025 11:48
@alamb alamb mentioned this pull request Mar 4, 2025
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

minor: SessionStateBuilder::with_default_features ergonomics

3 participants