Skip to content

Adding c10d ops and additional aten ops for DTensor and FSDP2#29

Open
komikndr wants to merge 8 commits intoComfy-Org:mainfrom
komikndr:main
Open

Adding c10d ops and additional aten ops for DTensor and FSDP2#29
komikndr wants to merge 8 commits intoComfy-Org:mainfrom
komikndr:main

Conversation

@komikndr
Copy link

@komikndr komikndr commented Feb 6, 2026

Preliminary pull request adding FP8 operators to support Raylight custom nodes.

The c10d operators will not be touched during a normal Comfy run, sincen as far as I know only Raylight uses DTensor and its operators, which require an initialized distributed process group. Normal ComfyUI/main.py execution does not initialize this.

The operators being added:

c10d

  • all_gather_into_tensor
  • broadcast_
  • wait_tensor
  • scatter_

aten

  • slice
  • split
  • new_zeros
  • as_strided
  • cat
  • alias

In practice, for c10d only all_gather_into_tensor and wait_tensor are currently required, since wait_tensor is needed for the FSDP unshard path where an in-place copy cannot directly transfer a QuantizedTensor into an unquantized tensor.

To use this inside a model, the nn.Module cannot inherit from a "disable init weight" operator, since every sharded parameter must be properly accounted for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant