EAGLE: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

up-to-date curated list of state-of-the-art Large vision language models hallucinations research work, papers & resources

Large Visual Language Model(LVLM), Large Language Model(LLM), Multimodal Large Language Model(MLLM), Alignment, Agent, AI System, Survey

A benchmark dataset and simple code examples for measuring the perception and reasoning of multi-sensor Vision Language models.

CLIP-MoE: Mixture of Experts for CLIP

LEMMA: An effective and explainable way to detect multimodal misinformation with LVLM and external knowledge augmentation, incorporating the intuition and reasoning capbility inside LVLM.

HiRED strategically drops visual tokens in the image encoding stage to improve inference efficiency for High-Resolution Vision-Language Models (e.g., LLaVA-Next) under a fixed token budget.

Code for USENIX Security 2024 paper: Moderating Illicit Online Image Promotion for Unsafe User Generated Content Games Using Large Vision-Language Models.

Novel approach that leverages LVLMs to efficiently generate high-quality synthetic VQA-NLE datasets.