Port SAM3 from inference/models to inference-models

[grzegorz-roboflow]

What does this PR do?

Add SAM3 to inference-models

Type of Change

• New feature (non-breaking change that adds functionality)

Testing

Added integration tests to cover SAM3

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code where necessary, particularly in hard-to-understand areas
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context

N/A

---

Note

Introduces SAM3 (Segment Anything 3) to inference-models with a PyTorch wrapper and caching utilities.

• Adds SAM3Torch with image embedding, point/box prompting, mask-input refinement, and text/visual prompting APIs; supports batching, logits return, and version validation
• New in-memory caches for image embeddings and low-res masks with LRU-like eviction; prior-prompt subset lookup optimizes interactive flows
• Adds entities (SAM3ImageEmbeddings, SAM3Prediction, SAM3MaskCacheEntry) and helpers (prompt serialization, mask selection, hashing)
• Documentation (README.md) with usage examples and a LICENSE.txt for SAM3
• Updates pyproject.toml to include sam3 and platform-conditional triton deps
• Extensive integration tests covering embeddings, segment APIs (points/boxes/masks), text prompting, batching, caching, and error cases
• Minor fix in legacy visual_segmentation.py to short-circuit cache search when prompt has ≤1 point

^{Written by Cursor Bugbot for commit bb1a8a2. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Shallow copy instead of deep copy mutates caller's data

High Severity

The pad_points function uses copy(args) (shallow copy) instead of copy.deepcopy(args) as in the original implementation. Since the function mutates nested lists via prompt.append([0, 0]) and label.append(-1), the caller's original point_coords and point_labels data structures are modified unexpectedly. This causes data corruption when the same input data is reused across multiple calls.

 

[cursor]

Numpy arrays not normalized like tensor inputs in embedding

Medium Severity

In _forward_image_embeddings, numpy arrays are used directly without the CHW-to-HWC transposition or 0-1 to 0-255 normalization that tensor inputs receive. However, segment_with_text applies both transformations to numpy and tensor inputs alike. This inconsistency means a float numpy array with values in [0,1] or CHW layout will produce incorrect embeddings when using embed_images/segment_images, but work correctly in segment_with_text.

 

[cursor]

Cache capacity check allows exceeding size limit

Low Severity

The _ensure_cache_has_capacity method uses while len(...) > self._size_limit but is called BEFORE adding a new item. This means when the cache has exactly size_limit items, nothing is removed, and the subsequent add causes the cache to exceed its limit by one item. The condition needs >= instead of > to properly make room before adding.

Additional Locations (1)

• inference_models/inference_models/models/sam3/cache.py#L152-L153