Indexing files is very slow

[tjkuson]

Expected behaviour

Downloading or validating files requires indexing the files first. This involves counting the files and their file size.

When I click accept to the prompt informing me that indexing the files will take a long time, I expect to see some sort of feedback.

Current behaviour

I click accept but see no feedback until the prompt saying that validating/saving files has started. This takes many seconds using the small dummy dataset, so could take a very long time for real datasets.

[tjkuson]

Possible approaches to this problem:

• Somehow speeding up the indexing process
• Making indexing of files happen behind-the-scenes
• Assign indexing to a worker with its own thread and monitor using a progress bar (like how it currently deals with saving and validating files)