Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(selectors): slice matcher fixes & tests #529

Merged
merged 4 commits into from
Jul 7, 2023
Merged

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Jun 28, 2023

Draft for now; this is fine but I need to make sure there's nothing else needed here for this piece of byte range queries (there's also more pieces).

  • Don't error slice matcher when not string or bytes, silently non-match
  • Cache offset in readerat so we don't over-all on the underlying read seeker
  • Add lots of tests, including various edges
  • Fix 32-bit breakage exposed by tests

@rvagg
Copy link
Member Author

rvagg commented Jun 29, 2023

I've also removed the hard error in the slice matcher when it reaches a node that's not a string or bytes; this is more consistent with approaches of most of the other selector pieces, they just silently ignore non-matching nodes.

rvagg added a commit to ipld/frisbii that referenced this pull request Jun 30, 2023
rvagg added a commit to ipld/frisbii that referenced this pull request Jun 30, 2023
rvagg added a commit to ipld/frisbii that referenced this pull request Jun 30, 2023
@rvagg rvagg changed the title fix: cache offsets for sequential reads fix(selectors): slice matcher fixes & tests Jul 4, 2023
@rvagg rvagg marked this pull request as ready for review July 4, 2023 06:36
@rvagg rvagg force-pushed the rvagg/slice-matcher-fixes branch from 55f3898 to 319ff19 Compare July 4, 2023 06:42
@rvagg
Copy link
Member Author

rvagg commented Jul 4, 2023

Ready for review now, supports basic byte ranges in Trustless Spec now as this is, sans negative ranges (next PR will add that). Summary:

  • Don't error slice matcher when not string or bytes, silently non-match
  • Cache offset in readerat so we don't over-all on the underlying read seeker
  • Add lots of tests, including various edges
  • Fix 32-bit breakage exposed by tests

traversal/selector/matcher_test.go Outdated Show resolved Hide resolved
traversal/selector/matcher_util.go Outdated Show resolved Hide resolved
@rvagg rvagg force-pushed the rvagg/slice-matcher-fixes branch from 319ff19 to 43637d8 Compare July 4, 2023 07:36
Copy link
Collaborator

@hannahhoward hannahhoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

acc += len(byts)
continue
}
n := copy(p, byts[mbnrs.offset-acc:])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dunno if this matters, but this fixture doesn't quite conform to the io.Reader interface, unless i'm mistaken. https://pkg.go.dev/io#Reader

Read is supposed to read between 0 and len(p) bytes. But if len(byts[mbnrs.offset-acc:]) > len(p) that won't be the case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy returns the number of bytes copied which is "the minimum of len(src) and len(dst)", so it's always going to be capped by len(p) in this code and we take the returned n as the authoritative offset increment. So I don't believe we're breaking the Reader contract here (unless I'm mistaken!).

@rvagg rvagg merged commit 349deb2 into master Jul 7, 2023
@rvagg rvagg deleted the rvagg/slice-matcher-fixes branch July 7, 2023 09:08
rvagg added a commit to ipld/frisbii that referenced this pull request Sep 2, 2023
rvagg added a commit to ipld/frisbii that referenced this pull request Sep 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants