From bdbaa002d28f131c1f258f69169992d8d101f0d2 Mon Sep 17 00:00:00 2001 From: Pablo Vela Date: Thu, 29 Aug 2024 06:36:41 -0500 Subject: [PATCH] add sam2_depthanything example (#7306) --- examples/manifest.toml | 1 + examples/python/sam2_depthanything/README.md | 24 ++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100644 examples/python/sam2_depthanything/README.md diff --git a/examples/manifest.toml b/examples/manifest.toml index e40a54a77fa1..759d0a0b7e66 100644 --- a/examples/manifest.toml +++ b/examples/manifest.toml @@ -77,6 +77,7 @@ examples = [ # display order, most interesting first "detect_and_track_objects", "face_tracking", + "sam2_depthanything", "human_pose_tracking", "gesture_detection", "ocr", diff --git a/examples/python/sam2_depthanything/README.md b/examples/python/sam2_depthanything/README.md new file mode 100644 index 000000000000..81faa503ed8d --- /dev/null +++ b/examples/python/sam2_depthanything/README.md @@ -0,0 +1,24 @@ + + + +https://vimeo.com/1003789426?loop=1&autopause=0&background=1&muted=1&ratio=2802:1790 + +## Background +Segment Anything 2 is follow up work on Segment Anything, that extends the state of the art segmentation capabilities into videos. This is done by adding a per session memory module that captures information about the target object in the video. This allows SAM 2 to track the selected object throughout all video frames, even if the object temporarily disappears from view, as the model has context of the object from previous frames. Depth Anything 2 is a monocular depth estimation model trained on a large amount of synthetic data and real data to achieve state of the art depth estimation. The two models are combined to allow tracking an object in 3D from just a single monocular video! + +## Run the code +This is an external example. Check the [repository](https://github.com/pablovela5620/sam2-depthanything) for more information. + +You can try the example on HuggingFace space [here](https://huggingface.co/spaces/pablovela5620/sam2-depthanything). + +It is highly recommended to run this example locally by cloning the above repo and running (make sure you have [Pixi](https://pixi.sh/latest/#installation) installed): +``` +git clone https://github.com/pablovela5620/sam2-depthanything.git +pixi run app +```