[feat] Add ostris / Flex.2-preview pipeline

[victolee0]

What does this PR do?

Fixes #11394 (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@yiyixuxu @asomoza

Example code

import torch
from diffusers import Flex2Pipeline
from diffusers.utils import load_image

name_or_path = "ostris/Flex.2-preview"

inpaint_image = load_image("https://ostris.com/wp-content/uploads/2025/04/dog.jpg")
inpaint_mask = load_image("https://ostris.com/wp-content/uploads/2025/04/dog_mask.jpg")
control_image = load_image("https://ostris.com/wp-content/uploads/2025/04/dog_depth.jpg")

dtype = torch.bfloat16

pipe = Flex2Pipeline.from_pretrained(
     name_or_path,
     torch_dtype=dtype
 ).to("cuda")

image = pipe(
     prompt="A white friendly robotic dog sitting on a bench",
     inpaint_image=inpaint_image,
     inpaint_mask=inpaint_mask,
     control_image=control_image,
     control_strength=0.5,
     control_stop=0.33,
     height=1024,
     width=1024,
     guidance_scale=3.5,
     num_inference_steps=50,
     generator=torch.Generator("cpu").manual_seed(42)
 ).images[0]
image.save(f"robot_dog.png")

Result:

Official Example:

[tin2tin]

Very cool. Haven't tested it yet. Does it automatically detect the type of control image is input (ex. OpenPose, canny or depth)?

[victolee0]

@tin2tin No, you must manually select the correct type that matches your specific input image.

[tin2tin]

How/where do you set that? In the ex. It's depth, but it could have been ex. Canny as well?

[nitinmukesh]

@tin2tin

It's very complex, took me 5 days. I just finished integrating, you can refer my repo for code tomorrow but it's custom pipeline based and you can replace with diffusers pipeline.

---

[nitinmukesh]

How/where do you set that? In the ex. It's depth, but it could have been ex. Canny as well?

You don't have to specify that, it automatically determines it.
control_image can have Line, Pose, Depth.

[victolee0]

Oh, I misunderstood the question. sry

[nitinmukesh]

@tin2tin

I have updated the code, you can refer.
https://github.com/newgenai79/sd-diffuser-webui/blob/main/modules/text2image/tab_flex2_preview.py

[victolee0]

cc @yiyixuxu @asomoza

[asomoza]

Hi @victolee0 and thanks for your contribution, I really like what @jaretburkett and Flex wants to do but the current model is an Early access preview release which means that we should probably wait for the full release.

Also I see two more issues with this PR:

• This pipeline is the custom pipeline that the author published with the model right? If it is, then the priority for adding this goes to the model author unless they don't want to.
• Flex uses Flux as a base (with some modifications) and probably it's just going to be just this pipeline which I'm not sure that needs a whole directory and maybe it can go in the Flux one. Not sure about this and also we're going to have the same problem with chroma. What do you think @yiyixuxu?