Research dLLM

Codebase for Diffusion Language Models Research

---

Research dLLM is a research-focused library for Diffusion Language Models (dLLMs), providing a comprehensive collection of baseline methods (primarily KV caching and decoding strategies) for reproducible experiments.

Why Use This Codebase?

• Unified Evaluation Framework: Research dLLM provides a standardized testing environment that allows users to seamlessly switch between different baseline methods.
• Clean and Well-Documented Code: Research dLLM is written with a strong emphasis on clarity and readability.
• Active and Ongoing Maintenance: Research dLLM is actively maintained and continuously updated. More out-of-the-box baselines will be included in the future.

News

[25/9/30] We released code reading guides, hoping this can help you to grasp our work :)

[25/10/13] Now, batch inference is supported!

[25/12/06] 🚀 We are looking for help to reproduce our experimental results using A100 GPUs. Please contact us if you can assist with the benchmarking.

Supported Methods

The corresponding usages can be found here.

KV Caching

Method	Paper	Original Code Repo
PrefixCache / DualCache
dLLM Cache
d2Cache		This Repo

Decoding Strategies

The corresponding usages can be found here.

Method	Paper	Original Code Repo
Vanilla / Semi-AR
Parallel
PC-Sampler
Certainty Prior Decoding		This Repo
DAEDAL
KLASS

Setup

# Create and activate the environment
conda create -n d2cache python=3.11 -y
conda activate d2cache

# Install dependencies
pip install -r requirements/common.txt

# Prepare dotenv file, and set model path manually 
cp .env.example .env

Evaluation

Please check docs/ for detailed instructions on how to run evaluations with different methods. A quick example is shown in srcipts/run_eval.sh.

Available models:

• llada-base: GSAI-ML/LLaDA-8B-Base
• llada-inst: GSAI-ML/LLaDA-8B-Instruct
• dream-base: Dream-org/Dream-v0-Base-7B
• dream-inst: Dream-org/Dream-v0-Instruct-7B

Available datasets:

• gsm8k
• humaneval / humaneval_instruct
• math-500
• mbpp / mbpp_instruct
• ... (all tasks specified in lm-eval are available)

Important

To evaluate humaneval and mbpp dataset on instruct fine-tuned models, please use their corresponding instruct variants, i.e., humaneval_instruct and mbpp_instruct for better post processing.

Additional general arguments can be specified in configs/generation/*.yaml or configs/gen_args.py.

Starchart

Citation

If you find d²Cache or this repository useful for your research and applications, please cite using this BibTeX:

@article{jiang2025d2cache,
  title={d $\^{} 2$ Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching},
  author={Jiang, Yuchu and Cai, Yue and Luo, Xiangzhong and Fu, Jiale and Wang, Jiarui and Liu, Chonghan and Yang, Xu},
  journal={arXiv preprint arXiv:2509.23094},
  year={2025}
}

Acknowledgment

We would like to thank the authors of all models and baseline methods for their excellent work and open-source contributions.

License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.