-
Notifications
You must be signed in to change notification settings - Fork 7.1k
Country dataset #5138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Country dataset #5138
Conversation
To addresses issue pytorch#5108.
To addresses issue pytorch#5108.
💊 CI failures summary and remediationsAs of commit f0883a5 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @puhuk, thanks a lot for the PR. I've added a few comments inline. There are three more issues:
- You used
find_classes
andmake_dataset
to create the dataset. This is far better than doing it yourself, but it would be even better to simply subclassImageFolderDataset
so you don't have to worry about that at all. Basically, you only need to handle the input parsing and download and simply passos.path.join(root, "country", self._split)
tosuper().__init__()
(along with the transforms of course). If you do that, you don't need to define__len__
and__getitem__
at all. - Could you add the dataset to our documentation? For that, simply add the class name to the list in
docs/source/datasets.rst
. - Could you add tests for the dataset? You can create a
Country211TestCase
in this file. You basically only need to provide a minimal configuration as well as provide fake data. You can read this docstring or look at the other test cases for examples. Due to the very easy structure, all the fake data can be created bydatasets_utils.create_image_folder
.
@pmeier Thank you for your kind review. Let me check and correct it soon :) |
Reflect code review
@pmeier I resend PR with your suggestion and test code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@puhuk I suggested some changes inline. After that, my comment for inheriting from ImageFolderDataset
(#5138 (review)) is still open. Let me know if you need help with that.
Don't worry about the merge conflicts yet, we can fix them before merge. Plus, there are some unrelated failing test that you also don't need to worry about.
Update with review
@pmeier I resend the PR with your review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@puhuk I pushed the changes to inherit from ImageFolder
. There are two things left to fix in the docstring. Afterwards this is good to go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pmeier Really thanks to your review and re-creation of my humble code. |
Summary: * Add Country211 dataset To addresses issue #5108. * Add Country211 dataset To addresses issue #5108. * Update country211.py * Update country211.py * Code review reflected Reflect code review * Update test_datasets.py * Update with review Update with review * inherit from ImageFolder * Update test/test_datasets.py * Docstring + minor test update Reviewed By: NicolasHug Differential Revision: D33618167 fbshipit-source-id: 04de3c5290b966ff97f21ea32b2f678079aa2a6c Co-authored-by: Philip Meier <github.pmeier@posteo.de> Co-authored-by: Nicolas Hug <nicolashug@fb.com>
cc @pmeier