[Research] Latent Perceptual Loss (LPL) for Stable Diffusion XL

[kashif]

What does this PR do?

An implementation of Latent Perceptual Loss (LPL) for training Stable Diffusion XL models, based on the paper "Boosting Latent Diffusion with Perceptual Objectives" (Berrada et al., 2025). LPL is a perceptual loss that operates in the latent space of a VAE, helping to improve the quality and consistency of generated images by bridging the disconnect between the diffusion model and the autoencoder decoder.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Thank you! Very clean stuff!

Should we

1. Credit the first author for helping with a reference implementation?
2. Should we apply LoRA on unet instead of full fine-tuning?
3. Change lpl_sdxl.py to train_sdxl_lpl.py?

Maybe update the PR description with a visual example from full fine-tuning run?

[sayakpaul]

Should also include a full representative training command. Currently, we only show LPL-specific bits in a command but lack one where a full-blown training can be launched (dataset_name, batch_size, etc.).

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[yiyixuxu]

oh thanks
sorry we let this go stale