perf: avoid repeated dependency scans in pip show

[KRRT7]

Fixes #14102.

pip show currently computes Required-by separately for each queried package.
For multiple queried packages, this repeatedly scans all installed distributions
and parses dependency metadata.

This PR avoids that repeated work. For a single queried package, it keeps a
direct lookup path and short-circuits dependency scans when possible. For
multiple queried packages, it builds the reverse dependency mapping once while
walking installed distributions, then reuses it for each queried package.

Benchmark

I used python -m timeit with a mocked installed environment containing 2,000
distributions and 1,000 dependents, varying the number of queried packages.

To reproduce, run this once on main and once on this PR branch:

for queries in 1 5 20 50; do
  echo "queries=$queries"
  SETUP=$(cat <<PY
from pip._internal.commands import show

class Metadata(dict):
    def get_all(self, key, default=None):
        return self.get(key, [] if default is None else default)

class Dependency:
    def __init__(self, name):
        self.name = name

class Distribution:
    def __init__(self, name, dependencies):
        self.canonical_name = name
        self.raw_name = name
        self.raw_version = "1.0"
        self.location = ""
        self.editable_project_location = None
        self.metadata = Metadata({"Name": name, "Summary": "", "Home-page": "", "Author": "", "Author-email": "", "License": "", "License-Expression": "", "Classifier": [], "Project-URL": []})
        self.metadata_version = "2.4"
        self.installer = "pip"
        self._dependencies = [Dependency(dep) for dep in dependencies]
    def iter_dependencies(self):
        return iter(self._dependencies)
    def iter_raw_dependencies(self):
        return iter(())
    def read_text(self, name):
        raise FileNotFoundError(name)
    def iter_declared_entries(self):
        return None

class Environment:
    def __init__(self, distributions):
        self._distributions = distributions
    def iter_all_distributions(self):
        return iter(self._distributions)

query_names = [f"target-{i}" for i in range($queries)]
distributions = [Distribution(name, [f"base-dep-{i % 5}"]) for i, name in enumerate(query_names)]
for i in range(2000 - $queries):
    deps = [f"noise-dep-{i % 25}"]
    if i < 1000:
        deps.append(query_names[i % $queries])
    distributions.append(Distribution(f"package-{i}", deps))
show.get_default_environment = lambda: Environment(distributions)
PY
)
  uv run python -m timeit -r 11 -n 50 -s "$SETUP" "list(show.search_packages_info(query_names))"
done

My latest results:

queried packages	main	PR
1	662 usec/loop	313 usec/loop
5	2.85 msec/loop	485 usec/loop
20	11.1 msec/loop	509 usec/loop
50	27.5 msec/loop	578 usec/loop

This keeps the common single-package case faster than main, while preserving
the expected scaling improvement for multiple queried packages.

[danielhollas]

If I understand correctly this PR regresses the perf for the case of single package which is unfortunate since that is supposedly a very common use case (certainly more common than querying >5 packages).

[KRRT7]

@danielhollas good point, made some changes

[uranusjr]

It feels unnecessary to me the single-name case needs a special case.

[KRRT7]

It feels unnecessary to me the single-name case needs a special case.

yeah, it does add a bit of complexity, though I was trying to regain the perf loss for the single package path that was mentioned earlier, I regained it and then some.

[2ykwang]

the mock reuses a small set of dep names, which is quite cache-friendly. real installs tend to have mostly unique ones. would it be possible to also benchmark against a real installed env? curious whether the speedup holds up there.

[KRRT7]

@2ykwang locally I did benchmark using my venv and system venvs, the speedup wasn't as big as in the body numbers, but still close enough.

[ichard26]

Tagging onto @uranusjr's comment, I don't believe it's worth the significant additional complexity for speeding up multi-package pip show invocations. They seem quite rare in practice and this is hardly a performance bottleneck. Even 5 packages at all once is an extreme scenario IMO.