Skip to content

Feature request: Filter argument to walkdir #46312

Open

Description

When traversing a large directory with sub-directories, there is currently no way of preventing the walkdir iterator recursing into an unwanted directory.
This comes up often:

  • Perhaps the directory is irrelevant, but 99% of the subdirectories reside there, so most the time is wasted traversing here
  • Perhaps a particular directory is mounted to an unresponsive drive/network, and so would stall the iterator for minutes if encountered
  • Perhaps the user does not have access to an irrelevant sub-directory and so it would throw an error, and the user does not want to change the onerror keyword, because the user actually does want to throw an error if the desired sub-directory is not accessible.

We could change it with a filter keyword, or alternatively as the first positional argument to allow do-syntax. Any subdirectory d for which the filter function f(d) === false would be skipped and never descended into.

I think it's warranted to special-case filtering for this iterator in particular, by adding it as an argument to the iterator instead of simply filtering the iterators output. Unlike other iterators, filtering an element away inside the iterator itself can seriously reduce the total number of iterations. And this iterator can have particularly annoying side effects if undesired elements are iterated over.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    filesystemUnderlying file system and functions that use it

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions