Skip to content

perf(ocr): reuse CTC argmax indices for confidence scores - #5223

Open
caitianxiao wants to merge 2 commits into
PaddlePaddle:developfrom
caitianxiao:perf/ctc-confidence-gather
Open

caitianxiao wants to merge 2 commits into
PaddlePaddle:developfrom
caitianxiao:perf/ctc-confidence-gather

Conversation

@caitianxiao

Copy link
Copy Markdown

CTCLabelDecode scans the vocabulary twice: once for character indices and again for confidence scores. Reuse the argmax indices with np.take_along_axis to avoid the second scan. Character decoding and word-box handling are unchanged. This is separate from the logits-copy change in #5182.

Added regression tests for float16/32/64, array layouts, list inputs, ties, blanks, repeated characters, nonfinite scores and word boxes. python -m pytest -q tests/test_ctc_label_decode.py: 29 passed. Black 24.4.2, the repository-selected Flake8 rules and license-header checks also passed. The tests exercise the decoder classes without initializing inference backends.

On a local CPU microbenchmark with NumPy 2.3.2 and float32 logits shaped (16, 128, 8192), median conversion/index/score-extraction time dropped from 10.05 ms to 9.09 ms across 20 measured pairs. This measures postprocessing, not end-to-end OCR.

Microbenchmark
import statistics
import time
import numpy as np

x = np.random.default_rng(17).random((16, 128, 8192), dtype=np.float32)
samples = {False: [], True: []}
for i in range(24):
    for gather in ([False, True] if i % 2 == 0 else [True, False]):
        start = time.perf_counter()
        p = np.array(x)
        idx = p.argmax(axis=-1)
        scores = (np.take_along_axis(p, idx[..., None], axis=-1)[..., 0]
                  if gather else p.max(axis=-1))
        elapsed = time.perf_counter() - start
        if i >= 4:
            samples[gather].append(elapsed)
for gather, times in samples.items():
    print("gather" if gather else "max", statistics.median(times) * 1000, "ms")

@CLAassistant

CLAassistant commented Oct 14, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants