Skip to content

Conversation

@MehulBatra
Copy link
Contributor

@MehulBatra MehulBatra commented Oct 5, 2023

Add support to perform an action before and after schema traversal in preorder fashion for schema
for object: structtype, maptype & listtype
Linked: #32

@MehulBatra MehulBatra changed the title add support of before and after in preorderschema visitor for traversal add support of before and after actions in preorderschema traversal Oct 5, 2023
@rdblue
Copy link
Contributor

rdblue commented Oct 5, 2023

I don't think that this is necessary and I think it also breaks future uses of the visitor.

Despite the name, the visitor this updates doesn't actually perform pre-order traversal. It creates futures that can be called to produce the result of further processing. In Java, we call this the "custom order" visitor because you can use it for any order you want by calling the future at a different point.

This PR changes to actual pre-order traversal. That's fine, but it means that you can't actually customize as much.

@Fokko Fokko changed the title add support of before and after actions in preorderschema traversal Support of before and after actions in preorderschema traversal Oct 13, 2023
@Fokko Fokko added this to the PyIceberg 0.6.0 release milestone Oct 13, 2023
@rdblue
Copy link
Contributor

rdblue commented Nov 5, 2023

Maybe it's me, but I don't understand the value of adding before and after callbacks to this visitor. A node's children are traversed when the future is called and that allows you to do whatever you want before and after further schema traversal. I think it makes more sense to consolidate the logic in the usual methods rather than use callbacks in this case.

@MehulBatra
Copy link
Contributor Author

Maybe it's me, but I don't understand the value of adding before and after callbacks to this visitor. A node's children are traversed when the future is called and that allows you to do whatever you want before and after further schema traversal. I think it makes more sense to consolidate the logic in the usual methods rather than use callbacks in this case.

It's still half baked I am working on it, but thanks for the feedback, will that into consideration.

@Fokko
Copy link
Contributor

Fokko commented Nov 5, 2023

Sorry for being so late here. This was suggested here: https://github.com/apache/iceberg/pull/7831/files#r1285259053 I'll leave it up to @rdblue to decide if he thinks this is valuable.

@Fokko
Copy link
Contributor

Fokko commented Mar 12, 2024

Let's keep it as is, and close this for now.

@Fokko Fokko closed this Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants