Skip to content

fix: normalize category labels in functional nominal metrics for non-zero-based labels - #3454

Open
xiaoyaoqilan wants to merge 6 commits into
Lightning-AI:masterfrom
xiaoyaoqilan:fix/nominal-label-normalization
Open

fix: normalize category labels in functional nominal metrics for non-zero-based labels#3454
xiaoyaoqilan wants to merge 6 commits into
Lightning-AI:masterfrom
xiaoyaoqilan:fix/nominal-label-normalization

Conversation

@xiaoyaoqilan

Copy link
Copy Markdown

Problem

Functional nominal metrics (cramers_v, pearsons_contingency_coefficient, theils_u, tschuprows_t) crash when category labels are not zero-based and contiguous:

cramers_v(torch.tensor([1, 1, 2, 2]), torch.tensor([1, 1, 2, 2]), bias_correction=False)
# RuntimeError: shape '[2, 2]' is invalid for input of size 7

Nominal association is invariant to renaming categories — labels [0,1] and [1,2] describe the same two categories and should produce the same result.

Root Cause

num_classes is inferred as the count of unique values (2), but the confusion-matrix updater uses raw label values as bin indices. Label 2 creates a bin outside the expected 2x2 range.

Fix

Added _normalize_categorical_labels to nominal/utils.py which maps observed labels to contiguous 0-based IDs when they aren't already. Called from all 4 update functions after NaN handling:

  • cramers.py
  • pearson.py
  • theils_u.py
  • tschuprows.py

Stateful metrics with explicit num_classes are unaffected (they already use stable encoding).

Verification

base = torch.tensor([0, 0, 1, 1])
shifted = base + 1
# Before: shifted input raises RuntimeError
# After: both produce tensor(1.) for Cramer's V

Fixes #3446

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Functional nominal metrics fail for non-zero-based category labels

1 participant