0.73.3 solves dependencies 1.64 times slower than 0.73.2

[nuqleo]

I have 5 seconds delay in solving deps when running 'dnf upgrade' with libdnf-0.73.3-1.fc39.x86_64. No delay after downgrading to libdnf-0.73.2-1.fc39.x86_64 from updates-archive.

[ppisar]

I confirm a 1.64 slowdown. There weren't many changes between 0.73.2 and 0.73.3:

fbd3447 (HEAD -> dnf-4-master, tag: 0.73.3, origin/dnf-4-master) Release 0.73.3
b245193 Fix a memory leak in glob_for_cachedir()
a6d89d6 Set pool flag to fix pool_addfileprovides_queue() without filelists.xml
5449211 Support colon in username, use LRO_USERNAME and LRO_PASSWORD
86bbb15 (tag: 0.73.2, jkolarik/dnf-4-master) Release 0.73.2

[jan-kolarik]

a6d89d6 Set pool flag to fix pool_addfileprovides_queue() without filelists.xml

This one looks suspicious to me.

[m-blaha]

Yeah, I did some experiments with/without a6d89d6 but I observed the slowdown only if the filelists metadata were loaded. (I added optional_metadata_types = filelists into dnf.conf).

[m-blaha]

See also #1673 for another side effect of this commit.

[kontura]

Interesting, I cannot reproduce it.
Does the size of the update matter? What libsolv version is used?

[m-blaha]

I did some testing with dnf5 and the latest libsolv (containing openSUSE/libsolv@081580d patch) and it looks like the performance drop caused by POOL_FLAG_ADDFILEPROVIDESFILTERED is still present:

❯ rpm -q libsolv
libsolv-0.7.30-20240926010211.6.g081580de.fc39.x86_64
❯ cat /etc/dnf/dnf.conf
[main]
optional_metadata_types = filelists

With current dnf5 HEAD (POOL_FLAG_ADDFILEPROVIDESFILTERED set):

❯ time ./dnf5/dnf5 upgrade --assumeno
...
Transaction Summary:
 Installing:         5 packages
 Upgrading:         24 packages
 Replacing:         24 package
 Removing:           5 packages

Total size of inbound packages is 140 MiB. Need to download 140 MiB.
After this operation, 40 KiB extra will be used (install 269 MiB, remove 269 MiB).
Operation aborted by the user.

real    0m3.940s
user    0m3.883s
sys     0m0.176s

With patched dnf5 to not use POOL_FLAG_ADDFILEPROVIDESFILTERED (I've commented out line https://github.com/rpm-software-management/dnf5/blob/9d57058dfd550ba9636341d0f7431f24046099c7/libdnf5/solv/pool.hpp#L91):

❯ time ./dnf5/dnf5 upgrade --assumeno
...
Transaction Summary:
 Installing:         5 packages
 Upgrading:         24 packages
 Replacing:         24 package
 Removing:           5 packages

Total size of inbound packages is 140 MiB. Need to download 140 MiB.
After this operation, 40 KiB extra will be used (install 269 MiB, remove 269 MiB).
Operation aborted by the user.

real    0m2.810s
user    0m2.798s
sys     0m0.125s

[kontura]

Thank you for the report.

Thanks to Marek I was able to reproduce this and track down the problem, it happens when there is a repository with a file dependency and filelists are enabled. To satisfy the dependency libsolv has to go through each file (and there is a lot of them). While this shouldn't happen, at least in newer fedoras, I do think it is more user friendly to restore the old behavior when filelists are used.

I made two PRs because we had this issue also in dnf5.

[m-blaha]

Both PRs have been merged, thank you!