Brush export to COCO Creates Extra Annotation for Detached or Occluded Masks

[Nonilas]

When using the brush tool to annotate a single object that is non-contiguous or partially occluded (e.g., a contrail behind a cloud), the software incorrectly creates an additional, separate annotation. Instead of grouping the multiple, detached masks under a single object ID, the tool erroneously generates a new, independent annotation for one of the segments. This leads to an incorrect count of object instances in the exported data, which can negatively impact model training.

To Reproduce
Steps to reproduce the behavior:

1. Go to the labeling interface in a project configured with BrushLabels.
2. Click on a brush label to select it for annotation.
3. Draw the first segment of the mask for a single object on the canvas.
4. To represent an occlusion, move the cursor to another area and draw the second, detached segment for the same object instance.
5. export the annotations to "Brush labels to COCO".
6. See error: The system creates an extra annotation instead of associating both drawn segments with a single annotation ID.
   Expected behavior
   The segmentations for a single, occluded object should be attributed to only one annotation instance. When a user draws multiple, detached brush strokes for the same label, these strokes should be stored as parts of a single complex region (e.g., a single annotation ID with multiple run-length encodings). The current behavior, where a detached annotation is incorrectly configured as a new, 3rd annotation, is erroneous. This issue arises specifically when annotating occluded objects that require multiple, non-contiguous masks to be fully represented.

Screenshots:

Caption: Annotating two contrail (green) with two detached brush strokes resulted in three separate annotations being created instead of the expected two. Bottom Left of the picture is the occlusion

Environment (please complete the following information):
OS: Containerized on Linux VM
Label Studio Version: 1.19
Additional context
This bug is critical for instance segmentation tasks where maintaining the correct object count is essential. The creation of "phantom" annotations for parts of a single object can introduce significant noise into the training dataset, leading the model to learn incorrect object priors and potentially degrading its performance on occluded objects. The expected output is a single annotation ID that contains the information for all its constituent, non-contiguous masks.

[heidi-humansignal]

Hello,
Thanks for flagging this. In Label Studio, each detached brush stroke creates its own region (and thus its own annotation ID) in the JSON. When you export to COCO, the converter treats each region independently, so multiple strokes on the “same” object become separate annotations. You can explore https://labelstud.io/tags/relations, this tag allows for relations.

Why this happens

• The BrushLabels tool generates one region per mouse-drag/release.

• There’s no built-in mechanism to merge non-contiguous strokes under a single region ID in the UI or export pipeline.

Workarounds

1. Post-process the JSON to merge strokes
   After exporting your project JSON, you can run a small script that:

• Groups all brush regions with the same label (and/or task)
• Decodes each region’s run-length encoding (RLE)
• Combines them via a logical OR
• Re-encodes the union mask back to RLE
• Emits one “merged” region and drops the originals

Thank you,
Abu

Comment by Abubakar Saad
Workflow Run