Leaderboard

• LVLM: ARC Prize, Chatbot, LLM Benchmark, OmniSpatial, SuperCLUE
• Detection: Zero-Shot COCO, Open-Voc COCO, Open-Voc LVIS
• Text-encoder: MTEB, SuperGLUE
• 6D Object Pose: BOP
• other: KITTI, Mip-NeRF 360

Installation

Install the basic requirements:

pip install -r requirements.txt

Select preferences and run the command to install PyTorch >= 2.5.1 locally.

• CLIP

pip install open_clip_torch

• CoDet

pip install mss git+https://github.com/facebookresearch/detectron2.git git+https://github.com/openai/CLIP.git

• Contact-GraspNet

pip install provider pyrender

• Depth Anything V2

pip install gradio_imageslider gradio==4.29.0

• Grounding DINO

pip install git+https://github.com/IDEA-Research/GroundingDINO.git

• Segment Anything V2

pip install git+https://github.com/facebookresearch/sam2.git

• Tag2Text

pip install fairscale git+https://github.com/xinyu1205/recognize-anything.git
wget https://huggingface.co/spaces/xinyu1205/recognize-anything/resolve/main/tag2text_swin_14m.pth?download=true
mv tag2text_swin_14m.pth?download=true checkpoints/tag2text_swin_14m.pth

• VGGT

pip install git+https://github.com/facebookresearch/vggt.git

• Zero123++

pip install diffusers

Web

Install the following packages to run the web server:

pip install fastapi uvicorn

You need to create a Python file that stores the API in a dictionary format within a variable named FUNCTIONS, and set the file path in server.py.

Then, run the server using the command below:

uvicorn server:app