-
Notifications
You must be signed in to change notification settings - Fork 317
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add sam2_depthanything example (#7306)
- Loading branch information
1 parent
9bebd1a
commit 3731b26
Showing
2 changed files
with
25 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
<!--[metadata] | ||
title = "Sam2 + DepthAnything2" | ||
tags = ["2D", "3D", "HuggingFace", "Depth", "Pinhole camera", "SAM", "Segmentation"] | ||
source = "https://github.com/pablovela5620/sam2-depthanything" | ||
thumbnail = "https://static.rerun.io/sam2_depthanything/ecc229c54a04c55bfba236f86e15cd285429e412/480w.png" | ||
thumbnail_dimensions = [480, 268] | ||
--> | ||
|
||
|
||
https://vimeo.com/1003789426?loop=1&autopause=0&background=1&muted=1&ratio=2802:1790 | ||
|
||
## Background | ||
Segment Anything 2 is follow up work on Segment Anything, that extends the state of the art segmentation capabilities into videos. This is done by adding a per session memory module that captures information about the target object in the video. This allows SAM 2 to track the selected object throughout all video frames, even if the object temporarily disappears from view, as the model has context of the object from previous frames. Depth Anything 2 is a monocular depth estimation model trained on a large amount of synthetic data and real data to achieve state of the art depth estimation. The two models are combined to allow tracking an object in 3D from just a single monocular video! | ||
|
||
## Run the code | ||
This is an external example. Check the [repository](https://github.com/pablovela5620/sam2-depthanything) for more information. | ||
|
||
You can try the example on HuggingFace space [here](https://huggingface.co/spaces/pablovela5620/sam2-depthanything). | ||
|
||
It is highly recommended to run this example locally by cloning the above repo and running (make sure you have [Pixi](https://pixi.sh/latest/#installation) installed): | ||
``` | ||
git clone https://github.com/pablovela5620/sam2-depthanything.git | ||
pixi run app | ||
``` |