[modular] add tests for qwen modular

[sayakpaul]

What does this PR do?

Some things came up with the tests that needed fixing. But LMK if you would like me to revert them.

I also fixed a bunch of small things related to tests as I was running them on both GPU and CPU.

Will propagate the changes from #12579 once it's merged.

[sayakpaul]

Similar to how it's done in the other pipelines.

[sayakpaul]

We have two options:

1. Either this
2. Or how it's done in edit:

   diffusers/src/diffusers/pipelines/qwenimage/pipeline_qwenimage_edit.py

   Lines 752 to 757 in 325a950

   	img_shapes = [
   	[
   	(1, height // self.vae_scale_factor // 2, width // self.vae_scale_factor // 2),
   	(1, calculated_height // self.vae_scale_factor // 2, calculated_width // self.vae_scale_factor // 2),
   	]
   	] * batch_size

Regardless, the current implementation isn't exactly the same as how the standard pipeline implements it and would break for the batched input tests we have.

[yiyixuxu]

this is for qwen image, not edit though https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py#L636

[sayakpaul]

My bad sorry. I meant to refer to the Qwen Image pipeline, actually:

diffusers/src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py

Line 636 in 5a47442

img_shapes = [[(1, height // self.vae_scale_factor // 2, width // self.vae_scale_factor // 2)]] * batch_size

In modular, we are doing:

diffusers/src/diffusers/modular_pipelines/qwenimage/before_denoise.py

Lines 544 to 553 in 5a47442

	block_state.img_shapes = [
	[
	(
	1,
	block_state.height // components.vae_scale_factor // 2,
	block_state.width // components.vae_scale_factor // 2,
	)
	]
	* block_state.batch_size
	]

Which I think are different (first just multiplies with batch_size while the second does an inner multiplication).

As a consequence of that, we get:

FAILED tests/modular_pipelines/qwen/test_modular_pipeline_qwenimage.py::TestQwenImageModularPipelineFast::test_inference_batch_consistent - RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 1
FAILED tests/modular_pipelines/qwen/test_modular_pipeline_qwenimage.py::TestQwenImageModularPipelineFast::test_inference_batch_single_identical - RuntimeError: The size of tensor a (16) must match the size of tensor b (32) at non-singleton dimension 1

The change in this PR fixes that. I also checked the test_qwenimage.py test-suite to see if we're skipping those failing tests and it seems like we are not. But I will defer to you for the final call here.

Let me know.

[yiyixuxu]

ahh ok got it!
let's just do same as the qwen image pipeline?

[sayakpaul]

Otherwise, no CFG settings would break.

[sayakpaul]

These are skipped in the standard pipeline tests, too.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

With transformers latest the modular tests are failing in this PR. Checking.

Update: Checking with the transformers team. Internal link:
https://huggingface.slack.com/archives/C014N4749J9/p1762245891630079

All of those tests work with transformers 4.57.1

[DN6]

Just comments on aligning with new testing structure.

[sayakpaul]

I will wait for your #12579 PR to get merged first. Easier that way for me to apply your suggestions.

[sayakpaul]

@DN6 I have aligned the tests to follow the current structure. PTAL.

[sayakpaul]

@DN6 the following tests are failing:

FAILED tests/modular_pipelines/stable_diffusion_xl/test_modular_pipeline_stable_diffusion_xl.py::TestSDXLModularPipelineFast::test_float16_inference - RuntimeError: expected scalar type Half but found Float
FAILED tests/modular_pipelines/stable_diffusion_xl/test_modular_pipeline_stable_diffusion_xl.py::TestSDXLImg2ImgModularPipelineFast::test_float16_inference - RuntimeError: expected scalar type Half but found Float

I think it's probably because of #12512

[yiyixuxu]

I'm not sure about img_shape here https://github.com/huggingface/diffusers/pull/12585/files#r2512902725

also, i think qwen pipelines currently do not support batch_size > 1, (num_images_per_prompt >1 is ok) so we can skip the test for multiple prompts for now

The rest of change looks good to me!

[sayakpaul]

@yiyixuxu perfect. I have also addressed your comment on vae_scale_factor.