Making large AI models cheaper, faster and more accessible

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Janus-Series: Unified Multimodal Understanding and Generation Models

YuE: Open Full-song Music Generation Foundation Model, something similar to Suno.ai but open

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

DeepSeek-VL: Towards Real-World Vision-Language Understanding

⚡ TabPFN: Foundation Model for Tabular Data ⚡

Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model

Chronos: Pretrained Models for Probabilistic Time Series Forecasting

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

EVA Series: Visual Representation Fantasies from BAAI

Images to inference with no labeling (use foundation models to train supervised models).

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

Emu Series: Generative Multimodal Models from BAAI

[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities