Repository navigation
LTX-2.5 SDR-To-HDR IC-LoRA: how should it be structured? #14981
Description
Activity
thanks for opening the issue!
it looks like something requires a new pipeline -> can you create & host a modular pipeline on hub for now? You can reuse some of the existing LTX2.5 blocks and only add what's specific to this Lora
some resources:
- Quickstart on modular diffusers (https://huggingface.co/docs/diffusers/modular_diffusers/quickstart)
- doc on building & share custom blocks (https://huggingface.co/docs/diffusers/modular_diffusers/custom_blocks)
- diffusers-cli related (https://huggingface.co/docs/diffusers/using-diffusers/cli#customblocks)
some examples:
https://huggingface.co/collections/diffusers/modular-pipelines
https://huggingface.co/collections/diffusers/modular-diffusers-custom-blocksplease share with us so we can help test and add into our collections
Thanks @yiyixuxu, that works for us. I'll build it as a modular pipeline on the Hub under scenario-labs, reusing the LTX-2.5 blocks and adding custom blocks for the SDR-To-HDR parts (ACEScct transforms, scene-embedding conditioning, seam keyframes). The keyframe-aware decoding changes the diffusion decoder itself, so I'll first ship the plain IC-LoRA path, then the seam keyframes with a custom decoder class, aligned with #14694 once it lands. I'll share the repo here when it's ready to test.
Reacted by YiYi XuThe first stage is up: https://huggingface.co/scenario-labs/ltx25-sdr-to-hdr-modular
It runs the SDR-To-HDR IC-LoRA (plain path, no seam keyframes yet) on stock
diffusersmain withtrust_remote_code=True. It reuses the LTX-2.5 modular blocks (LTX2InContextPrepareLatentsStep,LTX2ConditionSetTimestepsStep,LTX2ConditionPrepareCoordsStep, the condition denoise loop wrapper and its before/after steps,LTX2TrimConditionTokensStep) and adds seven blocks for the LoRA-specific parts: ACEScct input/output, the precomputed scene embedding instead of the text encoder, the first-frame keyframe marker, a placeholder audio token, a float32 decode with crop-back, and a loop denoiser that passesisolate_modalities=True. The stockLTX2LoopDenoisersetsisolate_modalitiesper guidance pass, so there was no way to request it from a block input; anisolate_modalitiesinput there would let us drop that custom denoiser.Validation: on a B200 with the real weights, three clips (720x480, a 1000x560 clip padded to 1024x576, and a 60 fps clip for the RoPE cap) give bitwise-identical latents and HDR output to a classic pipeline implementation of the same path, given the same input tensor.
One thing I hit while publishing: a Hub repo whose name contains a
.(e.g.LTX-2.5-...) can't use relative imports between its module files, becauseget_cached_module_filebuilds the module path aslocal/<org>--<repo>and Python reads the dot as a package separator. I renamed the repo to avoid it; happy to open a small PR if you'd like it handled.Seam keyframes are now in too: https://huggingface.co/scenario-labs/ltx25-sdr-to-hdr-modular (on by default,
seam_keyframes=Falsefor the plain path, plushigh_quality_hdr). The keyframe-aware decode ships askeyframe_decoder.py, a subclass ofLTX2VideoDiffusionDecoderModelloaded through the blocks'ComponentSpec, as an interim until diffusers supports it natively (Lightricks mentioned they are working on it). On a B200 with the real weights, 49- and 97-frame clips with seams,high_quality_hdrand a no-seam clip give bitwise-identical latents and HDR output to a classic implementation of the same path.
What
Lightricks/LTX-2.5-22b-IC-LoRA-SDR-To-HDRcan't run in diffusers today.LTX2HDRPipelinetargets the LTX-2.3 LogC3 LoRA. The LTX-2.5 one (Lightricks'ltx_pipelines.hdr_ic_lora.HDRICLoraPipeline) needs:decoder.type_emb, joint attention with the nearest keyframe planes).I opened #14966, #14974 and #14975 for this before discussing it here, sorry about that. I'm closing them and would like to agree on the shape first.
What I measured (H200, real weights,
hiker.mp4fromdocumentation-images, against the reference at LTX-29ec55f9f)Checkpoint blocker
The
diffusion_decoderinLightricks/LTX-2.5-Diffusersdoesn't match the current original VAE (ltx-2.5-video-vae): it has nodecoder.type_emb, and none of its decoder tensors match. Decoder-only parity against the reference is 44.4 dB (plain) / 35.2 dB (keyframes) with the published weights, and 62.2 / 68.6 dB once re-converted from the original file. Reported in https://huggingface.co/Lightricks/LTX-2.5-Diffusers/discussions/19. Keyframe decoding can't ship against the current Hub weights.Questions
LTX2HDRPipeline(hdr_transform="acescct", which changes the inputs, components, schedule and decoder), a separateLTX2SDRToHDRPipeline, or modular blocks?My suggested order, if that works for you: (a) ACEScct transforms + SDR-To-HDR without seams, in whatever shape you prefer; (b) the keyframe-aware decoder, after #14694 and the checkpoint fix; (c) seam keyframes in the pipeline; export separately or not at all.