Skip to content

Fix davies_bouldin_score returning a wrong value for off-origin float64 data - #3468

Open
VenishPaneliya wants to merge 2 commits into
Lightning-AI:masterfrom
VenishPaneliya:fix/3467-clustering-buffer-dtype
Open

Fix davies_bouldin_score returning a wrong value for off-origin float64 data#3468
VenishPaneliya wants to merge 2 commits into
Lightning-AI:masterfrom
VenishPaneliya:fix/3467-clustering-buffer-dtype

Conversation

@VenishPaneliya

Copy link
Copy Markdown

What does this PR do?

Fixes #3467

davies_bouldin_score and calinski_harabasz_score allocated their accumulators without a dtype, so the buffers were float32 whatever the input was. dunn_index — same family, same signature — preserves the input dtype, so the clustering metrics disagreed with each other.

For Davies-Bouldin this is a correctness bug, not a dtype-label one. Writing a float64 centroid into a float32 buffer rounds it to about seven significant digits, so cluster_k - centroids[k] ends up dominated by that rounding as soon as the data sits away from the origin:

data offset returned correct relative error
0 27.0118026733 27.0118054968 1.0e-07
1e3 27.0143089294 27.0118054967 9.3e-05
1e5 29.3704795837 27.0118054947 8.7e-02
1e7 1.7268073559 27.0118055903 9.4e-01

The clustering structure is identical in every row — only the origin moved. At offset=1e7 the metric returns 1.73 where the answer is 27.01.

Off-origin coordinates are ordinary: geographic positions, epoch timestamps, unnormalised sensor readings. A user reaching for float64 because their data has a large dynamic range had it silently discarded.

Calinski-Harabasz narrows only the running accumulation — the per-cluster terms are still computed in the input dtype — so its error stayed near float32 epsilon (~1e-7 relative). Still a silent float32 ceiling on a float64 input, but not catastrophic.

Changes

# davies_bouldin_score.py
- intra_dists = torch.zeros(num_labels, device=data.device)
- centroids = torch.zeros((num_labels, dim), device=data.device)
+ intra_dists = torch.zeros(num_labels, device=data.device, dtype=data.dtype)
+ centroids = torch.zeros((num_labels, dim), device=data.device, dtype=data.dtype)

# calinski_harabasz_score.py
- between_cluster_dispersion = torch.tensor(0.0, device=data.device)
- within_cluster_dispersion = torch.tensor(0.0, device=data.device)
+ between_cluster_dispersion = torch.tensor(0.0, device=data.device, dtype=data.dtype)
+ within_cluster_dispersion = torch.tensor(0.0, device=data.device, dtype=data.dtype)

Both degenerate branches also returned an explicit dtype=torch.float32 scalar; those now follow the input dtype too, so the return type is consistent on every path.

float32 inputs are unaffected — they already allocated float32 buffers, so no value moves.

Tests

New tests/unittests/clustering/test_clustering_dtype.py:

  • dtype preservation across all three data-based clustering metrics;
  • translation invariance for Davies-Bouldin and Calinski-Harabasz — shifting the data must not change a score defined purely by relative geometry. This is the test that pins the real regression;
  • Davies-Bouldin matched against an independent float64 implementation of the same formula;
  • class-based wrappers agreeing with the functional results;
  • float32 behaviour explicitly unchanged.

On the tolerances: subtracting a centroid from off-origin data cancels leading digits, so exact translation invariance is not achievable even in float64. Measured limits are 2.6e-12 at offset=1e3, 2.9e-10 at 1e5 and 5.2e-08 at 1e7, so the tolerance scales with the offset rather than pretending the error is zero. The bug produced a 94% error at 1e7, orders of magnitude outside those bounds, so the test still catches it decisively.

Verified the tests actually catch the bug — against unpatched src/:

$ pytest tests/unittests/clustering/test_clustering_dtype.py -q
10 failed, 12 passed in 6.13s

With the fix:

$ pytest tests/unittests/clustering/test_clustering_dtype.py -q
22 passed in 7.12s

Full clustering suite, compared against master in the same environment:

master : 10 failed, 187 passed, 8 skipped
this PR:  0 failed, 197 passed, 8 skipped

The 10 are exactly this PR's own tests. Nothing else in the suite moves.

Lint, types and doctests:

$ ruff check <changed files>                        All checks passed!
$ ruff format --check <changed files>               3 files already formatted
$ mypy <changed source files>                       Success: no issues found in 2 source files
$ pytest --doctest-modules <changed source files>   2 passed

Related

Same class of problem as #3465 / #3466 (spearman_corrcoef and kendall_rank_corrcoef), but a different mechanism — that one is integer division falling back to the default dtype, this one is untyped buffer allocation. Kept separate because the fix and the blast radius differ.

Before submitting

VenishPaneliya added a commit to VenishPaneliya/torchmetrics that referenced this pull request Aug 19, 2026
`davies_bouldin_score` and `calinski_harabasz_score` allocated their
accumulators without a `dtype`, so the buffers were float32 whatever the
input was. `dunn_index`, same family and same signature, preserves the
input dtype, so the clustering metrics disagreed with each other.

For Davies-Bouldin this was not only a dtype-label problem. Writing a
float64 centroid into a float32 buffer rounds it to roughly seven
significant digits, so `cluster_k - centroids[k]` is dominated by that
rounding once the data sits away from the origin. The score is then
wrong, not merely imprecise:

    offset   returned    correct     relative error
    0        27.011803   27.011805   1.0e-07
    1e3      27.014309   27.011805   9.3e-05
    1e5      29.370480   27.011805   8.7e-02
    1e7       1.726807   27.011806   9.4e-01

Off-origin coordinates are ordinary -- geographic positions, epoch
timestamps, unnormalised sensor readings -- and a user who reaches for
float64 because of a large dynamic range had it silently discarded.

Calinski-Harabasz narrows only the running accumulation, so its error was
bounded near float32 epsilon, but it was still a silent float32 ceiling
on a float64 input.

Both degenerate branches also returned an explicit float32 scalar; those
now follow the input dtype too.

Values are unchanged for float32 inputs, which already allocated float32
buffers.

Fixes Lightning-AI#3467
@VenishPaneliya
VenishPaneliya force-pushed the fix/3467-clustering-buffer-dtype branch from cc2256f to 657146c Compare August 24, 2026 10:16
@mergify mergify Bot removed the has conflicts label Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

davies_bouldin_score returns a wrong value for off-origin float64 data (float32 buffers)

1 participant