Downloader has synchronization issues

[anacrolix]

There are multiple race conditions and various inconsistencies in how torrents are added to the downloader.

Torrents can be added:

• Preexisting from disk via the server RPC
• New files to be converted to torrents and added via RPC
• Scanned from disk at startup by the downloader
• Added by filtering manifest files at startup with certain flags by the downloader

These all set configuration of the torrent differently, mostly failing to set data priorities or configure things like webseeds consistently which leads to apparent hangs. These can be identified more easily with #14637.

Other issues:

• Piece verification is racy due to using the wrong API and uses several nested worker pools and concurrency limits that don't play well together. Piece verification can also be wasteful as it can cause extra hashing to be queued.
• Torrent config is set conditionally based on various state. The torrent lib already handles this correctly based on priorities set.

Proposed solution:

• Use just the preverified file set passed through the RPC with consistent config.
• Fix piece verification concurrency.
• Add a hash verification method that avoids extra overhead.
• Create a more robust torrent storage implementation #14646
• Remove the download lock so syncing can continue after the initial sync.

[VBulikov]

anacrolix added Imp2 as Importance