Skip to content

Fix StreamingDataset.get_len(num_workers=0) #311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 6, 2024

Conversation

senarvi
Copy link

@senarvi senarvi commented Aug 6, 2024

This is how I think #310 could be fixed. This hasn't been discussed yet.

What does this PR do?

Fixes #310 .

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

Copy link

codecov bot commented Aug 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (main@777e8de). Learn more about missing BASE report.

Additional details and impacted files
@@          Coverage Diff          @@
##             main   #311   +/-   ##
=====================================
  Coverage        ?    78%           
=====================================
  Files           ?     34           
  Lines           ?   4931           
  Branches        ?      0           
=====================================
  Hits            ?   3836           
  Misses          ?   1095           
  Partials        ?      0           

Copy link
Collaborator

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Just for consistency.

@tchaton
Copy link
Collaborator

tchaton commented Aug 6, 2024

Hey @senarvi. Would you mind adding a test ?

Seppo Enarvi and others added 3 commits August 6, 2024 20:51
Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: thomas chaton <thomas@grid.ai>
@tchaton tchaton marked this pull request as ready for review August 6, 2024 18:15
@tchaton tchaton requested a review from awaelchli as a code owner August 6, 2024 18:15
@tchaton tchaton merged commit c60d559 into Lightning-AI:main Aug 6, 2024
34 checks passed
@tchaton
Copy link
Collaborator

tchaton commented Aug 6, 2024

Hey @senarvi. Thanks a lot for your contribution :)

@senarvi
Copy link
Author

senarvi commented Aug 7, 2024

Hey @senarvi. Would you mind adding a test ?

@tchaton I think adding a test is a good idea. You merged this pull request already, so I created a new one: #314

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

StreamingDataLoader sets num_workers to zero, which causes the dataset length to be reported incorrectly
2 participants