Restoring or moving torrents happens very slowly under unclear conditions

[Jerfer02]

qBittorrent & operating system versions

qBittorrent: 6.6.1 x64
Operating system: Windows 10 Pro 22H2 (10.0.19045) x64
Qt: 6.4.3
libtorrent-rasterbar: 1.2.19.0

What is the problem?

Soon after updating from v.4.3.9 to v4.6.1, I changed the resume storage type to SQLite and restarted. It took the same time to load as it did when I had v4.3.9 (~1 hour to load 170k torrents). Obviously, it restored from file rather than SQLite since it needed to add resume file data to the DB first.

Last night I closed qBittorrent for a restart and allowed it to completely quit in the background. After opening qBittorrent again, it initially started restoring torrents very quickly. 200-1400 torrents per second based on the log. But it gets weird.

Restoration starts being limited to exactly 100 and 150 torrents per second for several 1000s of torrents, and then the number starts steadily dropping randomly. Eventually, only 1-3 torrents are being restored per second. Toward the end, it steadily begins to up the number being restored, reaching about 36 torrrents per second at the end.

Restore start time: 2023-11-28 22:38:01
Restore end time: 2023-11-29 07:52:17
^9 hours 14 minutes 16 seconds

Now, the UI has a constant lag to it. My "average time in queue" had always been in the 800ms to 1200ms range. Now it's always around 6000ms. When I set a new destination and create a new folder, the characters lag as I type and after pressing enter, it takes about 20 seconds to register that the file was created. If I open Windows Explorer during this time, I see that the file had already been created (no lag in Windows Explorer), yet qBittorrent still acts like it's waiting for the folder to be created.

Today I noticed something strange and figured it'd be a hint worth reporting.

"C:\Program Files\qBittorrent" has always been my default save path, because it forces me to set an appropriate folder to save in when I start a torrent, since qBittorrent doesn't have permission to save in the folder. It errors out as I expect it to do.

5000 of my torrents had this save path. To debug, I decided to change the path of these torrents to "C:\Users\Surekey\Downloads\qBittorrent". At the time, about 300 torrents were unpaused (downloading/seeding/stalled) with appropriate save paths. I start the move.

Torrents are being moved at 20-50 torrrents per second. When I paused the 300 torrents to see if that would speed things up, the number being moved dropped to a constant 1-3 torrents per second. When I started the 300 torrents up again, it started moving torrents at 20-50 torrents per second again.

Steps to reproduce

I'll try to add steps to reproduce once I've restarted my computer and qBittorrent again, if it is in fact... reproducible. It takes over 9 hours to completely load qBittorrent if the problem isn't unfortunately solved with the restart.

Additional context

No response

Log(s) & preferences file(s)

No response

[Jerfer02]

Also, now the CPU never stops hovering around 16% with no torrents active:

The name in the task list says 4.3 MiB/s is being transferred but it's really 0 MiB/s. Just fyi.

[stalkerok]

Downloading 135176...
I would like to believe that you are not downloading them all at the same time. Also, I suspect the disk is very fragmented.

[Jerfer02]

The 135176 number are just those that haven't completed 100%. I don't think things got defragmented between v.4.3.9 and v4.6.1 within a week after upgrading. Keep in mind, this is the first time qBittorrent is running after resume data started being pulled from the DB vs. files. It never acted this way with v4.3.9 nor v4.6.1 when it was pulling resume data from files.

Unfortunately, it still hasn't exited on its own yet. CPU is now down to 12% from 16% and memory is slightly less:

Should I terminate or let it continue to see if it closes on its own?

[Jerfer02]

Later today, I'm going to import the date/times into a DB, count the number of torrents restored per second, and put it into a graph so we can look at the restore rate over time. It is very odd.

[stalkerok]

With so many torrents it is better to run the client minimized in tray and wait for all torrents to resume before deploying the interface, set the refresh interval to 30s (or more). Defragmenting the disk(s) will not hurt either.
Need a SQLite label for such issues.
Speed hovered in the header.

[Jerfer02]

Too many seconds to put on the X-axis for a graph (not useful), so here's the CSV file with date/times and the number of torrents restored per second extracted from my logs.

qBittorrent_Torrent_Restore_Rates.csv

In what order are torrents restored? Is it based on add date, random, etc?

[Jerfer02]

It still hasn't completely exited yet:

I'm terminating it. You all know this isn't normal, right? It's not defragmentation because restoring torrents doesn't recheck torrents. It just restores the torrent state (from the local SQLite DB on my SSD) and checks to see if files that were written to disk are missing based on whether or not the files within the torrent are marked for downloading.

I switched ResumeDataStorageType back to "Legacy" in qBittorrent.ini. I'll post a CSV file of the dates/times and restore rate after it's done loading.

[stalkerok]

I know it's not defragmentation, it's a SQLite DB issue, I'm just assuming based on your number of torrents that the disk is highly fragmented.

[Jerfer02]

Oh in that case, you're right! It's definitely defragmented, partly due to the choice to use Btrfs over ext4. Btrfs probably would've been fine if I could go back in time and enable the pre-allocate space option.

After restarting, it's at 42% loaded so far, 6 hours later. Looks like it's going to take as long or longer than it did previously since it's still loading from the DB (while extracting resume data to resume files at the same time). It'll take another restart after this one to load directly from resume files, but I'll definitely provide CSV files for this load and the next.

[Jerfer02]

I'm browsing through torrents.db looking around and most of the rows don't have a torrent name.

select count(case when name != '' then 1 end) AS NUM_POPULATED,
       count(case when name  = '' then 1 end) AS NUM_EMPTY,
       count(case when name IS NULL then 1 end) AS NUM_NULL
from   main.torrents;

NUM_POPULATED	NUM_EMPTY	NUM_NULL
45704     	126853        	454

It looks like the field only gets updated when the torrent name is modified, so maybe it's not a bug.

[Jerfer02]

It finished loading and only took 2 minutes longer than last time. So that's good. All progress was lost since the previous start, like hours worth of download progress and save path changes. If my resume data save interval is 1 minute, wouldn't those updates/changes be committed to the DB every minute? Most of my force rechecks ended up being stuck, too.

Start: 2023-11-30 17:29:30
End: 2023-12-01 02:46:15
^ 9 hours 16 minutes 45 seconds

qBittorrent_Torrent_Restore_Rates_from_DB_to_Resume_Files.csv

I just exited qBittorrent again to restart and load directly from resume files. I'll produce a CSV when it's done.

Update: It exited on its own!
(N) 2023-12-01T03:25:07 - qBittorrent termination initiated
(N) 2023-12-01T03:55:31 - qBittorrent is now ready to exit

[glassez]

@Jerfer02
I hope you understand that having 170K torrents in a session is not a common use case. There is not a single contributor who has a comparable number of torrents in their daily use to investigate this in detail. The only way to understand what really happens under such a load is the active participation of the affected people in testing it.
If you have the opportunity and desire to participate in this (using test builds from GitHub or building them yourself), then we could cooperate in an attempt to find weaknesses in this issue. Otherwise, all your reports are practically useless to solve a possible problem.

[Jerfer02]

Fair enough. This will be my last post on the issue but I wanted to at least leave you all with the CSV of restore rates for resume files (in my unique case). After multiple 9+ hour load sessions, the following wait time is nothing to me now, lol.

Start: (N) 2023-12-01T12:27:47
End: (N) 2023-12-01T12:48:21
^ 20 minutes 34 seconds

qBittorrent_Torrent_Restore_Rates_from_Resume_Files.csv

Thank you all.