Introduce Keypoint Models: YoloXPose & RTMO

[CuriousDolphin]

• update dependencies
• refactor NN
• add Darknet Backbone
• update backbone test
• modelling YoloXPose
• modelliing RTMO
• define keypoint output
• add Task.KEYPOINT
• add yoloxpose processor
  • preprocess
  • postprocess
  • export postprocess
• add RTMO processor
  • preprocess
  • postprocess
  • export postprocess
• add YoloXPose to registry
• rename loss names for both models with coherence with wandb
• check TORCHSCRIPT export
• handle skeleton on keypoint annotator
• add RTMO (s) to registry
• test yoloxpose train on coco-> loss dead with no apparent motivation
• find a fix for flip_map (maybe avoid to do flip augmentations in keypoint models)
• take skeleton from dataset do model_config
• test rtmo train on coco
• test rtmo on generic keypoint dataset
• test rtmo export and inference
• update visualizer
• upload RTMO pretrained weights on public repo
• add default keypoint augmentation
• update documentation and examples

Key Changes

✨ Introduce keypoints models

• add RTMO-S/M/L-COCO keypoint pretrained model
  example:

from focoos import ModelManager
from PIL import Image

im = "https://public.focoos.ai/samples/federer.jpg"
model = ModelManager.get("rtmo-s-coco")
detections = model.infer(im,annotate=True, threshold=0.5)
Image.fromarray(detections.image) # visualise or save annotated image

📷 Unified Inference API

Standardize infer Method Signatures

• consistent infer() method across FocoosModel, InferModel, and RemoteModel with unified parameters: infer(image, threshold=0.5, annotate=False) and use unified image loader for infer methods (with also remote image support)
• add default threshold to 0.5
• Remove dependency on external annotate_image() function calls
• Streamlined workflow: get detections and visual annotations in a single call

example torch and exported model:

from focoos import ModelManager, RuntimeType
from PIL import Image


im = "https://public.focoos.ai/samples/motogp.jpg" # remote image, can also be local path, numpy array, or PIL image
model = ModelManager.get("fai-detr-l-obj365")
detections = model.infer(im,annotate=True, threshold=0.5) # annotatate param 
# Image.fromarray(detections.image) # visualise or save annotated image

# export model
model = model.export(RuntimeType.ONNX_CUDA32)
res = model.infer(im, annotate=True, threshold=0.5)

Image.fromarray(detections.image) # visualise or save annotated image

example with remote inference:

from focoos import FocoosHUB
from PIL import Image

hub = FocoosHUB()
model_ref = "fai-detr-l-obj365" # use any of pretrained model on app.focoos.ai or your own model reference
remote_model = hub.get_remote_model(model_ref)

im = "https://public.focoos.ai/samples/federer.jpg"

detections = remote_model.infer(im,annotate=True, threshold=0.5)

Image.fromarray(detections.image) # visualise or save annotated image

Enhanced FocoosDetections Structure

• add new image field: stores annotated results as base64 string or numpy array
• migrated from Pydantic to pure Python dataclasses for better performance,
  Improved serialization and memory usage
• add new keypoints field
• add pprint and print_infer methods to unify detections prints.

⌨️ CLI

• add new CLI command: focoos gradio to launch a Gradio interface for image and video inference using Focoos pretrained models.

🕹️ Trainer

• fix missing model preprocessing when amp=True (Automatic Mixed Precision) is enabled
• add COSINE scheduler quadratic warmup
• add KeypointEvaluator
• enhance logging with additional info
• Update Visualizer (preview hook) to save RGB images instead of BGR
• Restore TensorBoard Hook

📖 ModelRegistry

• model registry now support automatic loading json configs from registry folder instead of declare model configs manually

🏞️ Processor

• add image_size into init instead of preprocess methods
• improve image loader performance
• add non-blocking image transfer
• optimize preprocessor speed
• add focoos palette to annotators

📖 Docs

• add RTMO docs
• update Readme, Docs and notebook with from focoos import x for all exported classes and functions instead of absolute path

[github-actions]

Tests	Skipped	Failures	Errors	Time
228	0 💤	0 ❌	0 🔥	20.063s ⏱️