Changing resolution does not work

[leszko]

Summary

When the resolution is changed, then the stream fails to start with this error:

Traceback (most recent call last):
  File "/home/user/scope/src/scope/server/frame_processor.py", line 285, in process_chunk
    output = pipeline(**call_params)
  File "/home/user/scope/src/scope/core/pipelines/longlive/pipeline.py", line 163, in __call__
    return self._generate(**kwargs)
  File "/home/user/scope/src/scope/core/pipelines/longlive/pipeline.py", line 186, in _generate
    _, self.state = self.blocks(self.components, self.state)
  File "/home/user/scope/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/user/scope/.venv/lib/python3.10/site-packages/diffusers/modular_pipelines/modular_pipeline.py", line 917, in __call__
    pipeline, state = block(pipeline, state)
  File "/home/user/scope/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/user/scope/src/scope/core/pipelines/wan2_1/blocks/denoise.py", line 163, in __call__
    _, denoised_pred = components.generator(
  File "/home/user/scope/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/user/scope/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/user/scope/src/scope/core/pipelines/wan2_1/components/generator.py", line 263, in forward
    pred_x0 = self._convert_flow_pred_to_x0(
  File "/home/user/scope/src/scope/core/pipelines/wan2_1/components/generator.py", line 152, in _convert_flow_pred_to_x0
    x0_pred = xt - sigma_t * flow_pred

Platform

Ubuntu

Nvidia GPU

NVIDIA 5090

Scope Version

main

uv Version

No response

node Version

No response

[yondonfu]

I don't think this behavior was present before the modular pipeline refactor. I think there used to be rounding logic when setting height and width that ensured that the values were divisible by a constant factor required by the model. This logic can be found by just navigating to a commit prior to merging the modular pipeline refactor PR.

If this turns out to be the problem worth considering what the right behavior is here. It is questionable for the pipeline to automatically doing the rounding of output dimensions here - it is more convenient but can lead to unexpected/confusing behavior eg setting a specific output resolution and getting a slightly different resolution. Might be better to have validation requiring height and width to be divisible by the specific constant factor and then UI can just increment/decrement by that value when setting height and width and also do that validation?

[leszko]

After thinking about it a bit, I think that the pipeline should not automatically adjust the resolution, because it's somehow weird that it automatically scales the resolution. Also it raises a question where we should actually "scale" it, input resolution? Latents? etc.

I ended up implementing it in the frontend only. Also, it will be consistent with the Daydream API integration where we also don't allow passing any arbitrary resolution (FYI: @mjh1 ).

resolution.mp4