Description
When traversing a large directory with sub-directories, there is currently no way of preventing the walkdir
iterator recursing into an unwanted directory.
This comes up often:
- Perhaps the directory is irrelevant, but 99% of the subdirectories reside there, so most the time is wasted traversing here
- Perhaps a particular directory is mounted to an unresponsive drive/network, and so would stall the iterator for minutes if encountered
- Perhaps the user does not have access to an irrelevant sub-directory and so it would throw an error, and the user does not want to change the
onerror
keyword, because the user actually does want to throw an error if the desired sub-directory is not accessible.
We could change it with a filter
keyword, or alternatively as the first positional argument to allow do
-syntax. Any subdirectory d
for which the filter function f(d) === false
would be skipped and never descended into.
I think it's warranted to special-case filtering for this iterator in particular, by adding it as an argument to the iterator instead of simply filtering the iterators output. Unlike other iterators, filtering an element away inside the iterator itself can seriously reduce the total number of iterations. And this iterator can have particularly annoying side effects if undesired elements are iterated over.