Diffusion Models in Vision: A Survey

Denoising diffusion models represent a recent emerging topic in computer vision, demonstrating remarkable results in the area of generative modeling. A diffusion model is a deep generative model that is based on two stages, a forward diffusion stage and a reverse diffusion stage. In the forward diffusion stage, the input data is gradually perturbed over several steps by adding Gaussian noise. In the reverse stage, a model is tasked at recovering the original input data by learning to gradually reverse the diffusion process, step by step. Diffusion models are widely appreciated for the quality and diversity of the generated samples, despite their known computational burdens, i.e. low speeds due to the high number of steps involved during sampling. This repository categorizes the papers about diffusion models, applied in computer vision, according to their target task. The classifcation is based on our survey Diffusion Models in Vision: A Survey

Summary

1. Unconditional Generation
2. Conditional Generation
3. Text-to-Image generation
4. Super-Resolution
5. Image Editing
6. Region Image Editing
7. Inpainting
8. Image-to-Image Translation
9. Image Segmentation
10. Multi-Task
11. Medical Image-to-Image Translation
12. Medical Image Generation
13. Medical Image Segmentation
14. Medical Image Anomaly Detection
15. Video Generation
16. Few-Shot Image Generation
17. Counterfactual Explanations and Estimations
18. Image Restoration
19. Image Registration
20. Adversarial Purification
21. Semantic Image Generation
22. Shape Generation and Completion
23. Classification
24. Point Cloud Generation
25. Theoretical

Content

Unconditional Generation

1. Deep unsupervised learning using non-equilibrium thermodynamics
2. Denoising diffusion probabilistic models
3. Improved techniques for training score-based generative models
4. Adversarial score matching and improved sampling for image generation
5. Maximum likelihood training of score-based diffusion models
6. D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation
7. Diffusion Normalizing Flow
8. Diffusion Schrodinger bridge with applications to score-based generative modeling
9. Structured denoising diffusion models in discrete state-spaces
10. Score-based generative modeling in latent space
11. Improved denoising diffusion probabilistic models
12. Denoising Diffusion Implicit Models
13. Non-Gaussian denoising diffusion models
14. Bilateral denoising diffusion models
15. Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes
16. Noise estimation for generative diffusion models
17. Gotta go fast when generating data with score-based models
18. Learning to efficiently sample from diffusion probabilistic models
19. Deep generative learning via Schrodinger bridge
20. VAEs meet Diffusion Models: Efficient and High-Fidelity Generation
21. Variational diffusion models
22. Score-based generative modeling with critically-damped Langevin diffusion
23. Tackling the generative learning trilemma with Denoising Diffusion GANs
24. Heavy-tailed denoising score matching
25. Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models
26. Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality
27. Truncated Diffusion Probabilistic Models
28. Subspace Diffusion Generative Models
29. Maximum Likelihood Training of Implicit Nonlinear Diffusion Models
30. On Analyzing Generative and Denoising Capabilities of Diffusion-based Deep Generative Models
31. Diffusion-GAN: Training GANs with Diffusion
32. Accelerating Score-based Generative Models for High-Resolution Image Synthesis

Conditional Generation

1. Diffusion models beat gans on image synthesis
2. Classifier-Free Diffusion Guidance
3. On Fast Sampling of Diffusion Probabilistic Models
4. DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents
5. Pseudo Numerical Methods for Diffusion Models on Manifolds
6. Cascaded Diffusion Models for High Fidelity Image Generation
7. High Fidelity Visualization of What Your Self-Supervised Representation Knows About
8. Itô-Taylor Sampling Scheme for Denoising Diffusion Probabilistic Models using Ideal Derivatives
9. {Dynamic Dual-Output Diffusion Models
10. Generating High Fidelity Data from Low-density Regions using Diffusion Models
11. Perception Prioritized Training of Diffusion Models
12. Elucidating the Design Space of Diffusion-Based Generative Models
13. Progressive distillation for fast sampling of diffusion models
14. Denoising Likelihood Score Matching for Conditional Score-based Data Generation
15. On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models
16. A Continuous Time Framework for Discrete Denoising Models
17. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
18. Compositional Visual Generation with Composable Diffusion Models

Text-to-Image generation

1. Vector quantized diffusion model for text-to-image synthesis
2. Hierarchical text-conditional image generation with CLIP latents
3. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
4. Fast Sampling of Diffusion Models with Exponential Integrator
5. DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder
6. Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
7. Text2Human: Text-Driven Controllable Human Image Generation
8. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

Super-Resolution

1. Image super-resolution via iterative refinement
2. Score-based Generative Neural Networks for Large-Scale Optimal Transport

Image Editing

1. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
2. Blended Latent Diffusion

Region Image Editing

1. Blended diffusion for text-driven editing of natural images

Inpainting

1. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
2. RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Image-to-Image Translation

1. Palette: Image-to-Image Diffusion Models
2. UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models
3. EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations
4. Pretraining is All You Need for Image-to-Image Translation
5. VQBB: Image-to-image Translation with Vector Quantized Brownian Bridge
6. The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Models

Image Segmentation

1. Label-Efficient Semantic Segmentation with Diffusion Models
2. SegDiff: Image Segmentation with Diffusion Probabilistic Models

Multi-Task

1. Generative modeling by estimating gradients of the data distribution
2. Score-Based Generative Modeling through Stochastic Differential Equations
3. ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
4. Learning Energy-Based Models by Diffusion Recovery Likelihood
5. Conditional image generation with score-based diffusion models
6. More control for free! Image synthesis with semantic diffusion guidance
7. ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models
8. Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generation
9. High-Resolution Image Synthesis with Latent Diffusion Models
10. Diffusion Autoencoders: Toward a Meaningful and Decodable Representation
11. Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction
12. DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
13. Understanding DDPM Latent Codes Through Optimal Transport
14. Conditional Simulation Using Diffusion Schrödinger Bridges
15. Retrieval-Augmented Diffusion Models
16. Accelerating Diffusion Models via Early Stop of the Diffusion Process
17. Diffusion Models as Plug-and-Play Priors
18. Non-Uniform Diffusion Models

Medical Image-to-Image Translation

1. Unsupervised Medical Image Translation with Adversarial Diffusion Models
2. Unsupervised Denoising of Retinal OCT with Diffusion Probabilistic Model

Medical Image Generation

1. Solving inverse problems in medical imaging with score-based generative models
2. Score-based diffusion models for accelerated MRI

Medical Image Segmentation

1. Diffusion Models for Implicit Image Segmentation Ensembles

Medical Image Anomaly Detection

1. Diffusion Models for Medical Anomaly Detection
2. Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models
3. AnoDDPM: Anomaly Detection With Denoising Diffusion Probabilistic Models Using Simplex Noise
4. What is Healthy? Generative Counterfactual Diffusion for Lesion Localization

Video Generation

1. Video Diffusion Models
2. Diffusion Probabilistic Modeling for Video Generation
3. Flexible Diffusion Modeling of Long Videos
4. Diffusion Models for Video Prediction and Infilling

Few-Shot Image Generation

1. Few-Shot Diffusion Models

Counterfactual Explanations and Estimations

1. Diffusion Models for Counterfactual Explanations
2. Diffusion Causal Models for Counterfactual Estimation

Image Restoration

1. Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models
2. Denoising Diffusion Restoration Models

Image Registration

1. DiffuseMorph: Unsupervised Deformable Image Registration Along Continuous Trajectory Using Diffusion Models

Adversarial Purification

1. Diffusion Models for Adversarial Purification

Semantic Image Generation

1. Semantic Image Synthesis via Diffusion Models

Shape Generation and Completion

1. 3D shape generation and completion through point-voxel diffusion

Classification

1. Score-based generative classifiers

Point Cloud Generation

1. Diffusion Probabilistic Models for 3D Point Cloud Generation

Theoretical

1. A variational perspective on diffusion-based generative models and score matching