Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphsync Simplication Idea: Refactor go-ipld-prime selector traversals so we don't need ipldutil.Traverser #260

Open
hannahhoward opened this issue Oct 26, 2021 · 0 comments
Labels
effort/days Estimated to take multiple days, but less than a week kind/discussion Topical discussion; usually not changes to codebase

Comments

@hannahhoward
Copy link
Collaborator

What

Rather than use ipldutil.Traverser, refactor go-ipld-primes selector traversal code so you can directly interact at the block level.

Why was it this way

Go-ipld-prime's selector traversal returns information -- assembled ipld nodes -- that are unrelated to what we want to send over the wire -- IPLD blocks. As a result, all of the logic to send and receive data happens inside of a callback you pass to the selector traversal for block loading.

Consequences

We do a lot to work around the fact that the interface for go-ipld-prime's selector traversal fundamentally does not meet the needs of go-graphsync. We've ended up with a complex module called the Traverser that masks the actual structure of the ipld-prime traversal call inside of an interface that is slightly more suited to the needs of go-graphsync. It contains a hidden go-routine and while the channel logic works, it's quite complicated and has needed several revisions.

Why leave it as it is

To some extent, refactoring selector traversal would just move the complexity to go-ipld-prime. However, if we can built a version with no go-routines and less recursion,specifically suited to the needs of go-graphsync, this would make debugging much easier

@hannahhoward hannahhoward added need/triage Needs initial labeling and prioritization effort/days Estimated to take multiple days, but less than a week kind/discussion Topical discussion; usually not changes to codebase and removed need/triage Needs initial labeling and prioritization labels Oct 26, 2021
@hannahhoward hannahhoward moved this from Icebox to Backlog in Project Thunder (Interop) Nov 18, 2021
@hannahhoward hannahhoward moved this from Backlog to Icebox in Project Thunder (Interop) Nov 18, 2021
@hannahhoward hannahhoward moved this from Icebox to Backlog in Project Thunder (Interop) Nov 18, 2021
@hannahhoward hannahhoward moved this from Backlog to Icebox in Project Thunder (Interop) Jan 25, 2022
@mvdan mvdan removed their assignment Mar 30, 2022
marten-seemann pushed a commit that referenced this issue Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/days Estimated to take multiple days, but less than a week kind/discussion Topical discussion; usually not changes to codebase
Projects
None yet
Development

No branches or pull requests

2 participants