Multi-controlnet batching for StableDiffusionXLControlNetInpaintPipeline

[kdubovikov]

Is your feature request related to a problem? Please describe.
Currently, batching is not supported when we are conditioning the SDXL pipeline on multiple controlnets:

diffusers/src/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint_sd_xl.py

Line 771 in 28f9d84

raise ValueError("A single batch of multiple conditionings are supported at the moment.")

Describe the solution you'd like.
We can pass a list of lists of images for batch inference.

Describe alternatives you've considered.
One-by-one processing, which results in very low GPU utilization.

There was a similar PR raised for the StableDiffusionControlNetPipeline #6334

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.