GLiNER.inference() (and the lower-level prepare_batch → collate_batch → run_batch → decode_batch path used internally, including by gliner.serve) documents/accepts per-text label lists via labels: List[List[str]]. This works when every item in the batch has the same number of labels, but crashes with a KeyError in decoder.py:_build_span_tuple whenever items in the same batch have different label counts.
Environment
gliner==0.2.27- Python 3.13, macOS (also reproduced via
gliner serve / Ray Serve on the same machine)
Repro
from gliner import GLiNER
model = GLiNER.from_pretrained("urchade/gliner_small-v2.1")
texts = [
"Tim Cook said Apple would expand its stores in India.",
"The Bank of England raised rates; sterling fell against the Swiss franc.",
]
labels = [
["person", "company"], # 2 labels
["currency"], # 1 label -- different COUNT
]
model.inference(texts, labels, threshold=[0.5, 0.3], flat_ner=True, multi_label=True)Expected
Per-text entities, same as calling predict_entities() once per text with its own labels/threshold.
Actual
File ".../gliner/decoding/decoder.py", line 562, in _build_span_tuple
ent_type = id_to_class[class_idx + 1] # +1 because 0 is <pad>
KeyError: 2
Notes
- Same label count, different content (e.g.
["currency", "company"] vs ["company", "exchange"]) batches fine — the bug is specifically about count mismatch, not content.
- A single request at a time (no batching) always works, regardless of label count.
- Reproduces identically through
GLiNERServer._run_batch_internal and through a live gliner serve deployment (python -m gliner.serve) under real concurrent HTTP requests — Ray's @serve.batch collates concurrent requests with different label counts into one physical batch, which then crashes the same way (surfaced as a 404 {"error":"2"} since the handler's except KeyError maps it to 404).
This breaks the documented capability in gliner/serve/server.py's _infer_batch docstring: "Supports heterogeneous request parameters by passing per-text labels... through to the model decode path."
GLiNER.inference()(and the lower-levelprepare_batch → collate_batch → run_batch → decode_batchpath used internally, including bygliner.serve) documents/accepts per-text label lists vialabels: List[List[str]]. This works when every item in the batch has the same number of labels, but crashes with aKeyErrorindecoder.py:_build_span_tuplewhenever items in the same batch have different label counts.Environment
gliner==0.2.27gliner serve/ Ray Serve on the same machine)Repro
Expected
Per-text entities, same as calling
predict_entities()once per text with its own labels/threshold.Actual
Notes
["currency", "company"]vs["company", "exchange"]) batches fine — the bug is specifically about count mismatch, not content.GLiNERServer._run_batch_internaland through a livegliner servedeployment (python -m gliner.serve) under real concurrent HTTP requests — Ray's@serve.batchcollates concurrent requests with different label counts into one physical batch, which then crashes the same way (surfaced as a404 {"error":"2"}since the handler'sexcept KeyErrormaps it to 404).This breaks the documented capability in
gliner/serve/server.py's_infer_batchdocstring: "Supports heterogeneous request parameters by passing per-text labels... through to the model decode path."