Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort files in DirectoryIterator because os.walk orders them arbitrarily #7869

Merged
merged 1 commit into from
Sep 11, 2017

Conversation

JakeBecker
Copy link
Contributor

@JakeBecker JakeBecker commented Sep 10, 2017

The filenames yielded by the DirectoryIterator are in arbitrary order because it uses os.walk (which uses os.listdir under the hood. The docs for os.listdir mention that the filenames are ordered arbitrarily: https://docs.python.org/3/library/os.html#os.listdir . On Mac OSX, the filenames seem to be sorted lexicographically, but on Ubuntu 17.04 with ext4 filesystem, they seem to be given in an arbitary, non-lexicographic order.

The Keras docs give an example of combining images with masks using flow_from_directory, which requires that the files be sorted: https://keras.io/preprocessing/image/ . This fix is important for that to work on all systems.

Fixes #6418

Copy link
Member

@fchollet fchollet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fchollet fchollet merged commit 419105b into keras-team:master Sep 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DirectoryIterator should have a sorted filenames when shuffle=False
2 participants