Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add filter/keep by http status to RecordLoader class #315

Closed
greebie opened this issue Apr 5, 2019 · 0 comments · Fixed by #347
Closed

Add filter/keep by http status to RecordLoader class #315

greebie opened this issue Apr 5, 2019 · 0 comments · Fixed by #347

Comments

@greebie
Copy link
Contributor

greebie commented Apr 5, 2019

Is your feature request related to a problem? Please describe.

Now that we have http status as part of the ArchiveRecord class, it may make sense to include methods to filter and or keep record with particular status codes.

Describe the solution you'd like
Something like:

RecordLoader.loadArchives(path, sc)
  .keepValidPages()
  .filterHttpStatus(Set('404', '410'))

and same for .keepHttpStatus

Describe alternatives you've considered
@edsu also suggested that users may already expect certain Http Statuses (e.g. 410) to be removed from .keepValidPages() as well.

Additional context

See discussion in ArchivesUnleashed Slack Aut Channel.

@ruebot ruebot self-assigned this Aug 17, 2019
ruebot added a commit that referenced this issue Aug 17, 2019
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315
ruebot added a commit that referenced this issue Aug 17, 2019
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315
ruebot added a commit that referenced this issue Aug 17, 2019
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315
ianmilligan1 pushed a commit that referenced this issue Aug 17, 2019
- Add keep and discard by http status RecordLoader
- Add tests
- Clean up/add doc comments in RecordLoader
- Resolve #315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants