Skip to content

Load workflows from mp4/mov files#3543

Merged
christian-byrne merged 7 commits intomainfrom
parse-mp4
Apr 26, 2025
Merged

Load workflows from mp4/mov files#3543
christian-byrne merged 7 commits intomainfrom
parse-mp4

Conversation

@christian-byrne
Copy link
Contributor

@christian-byrne christian-byrne commented Apr 21, 2025

Resolves #3538 by supporting loading workflows from mp4 files. Also allows parsing .mov and .m4v, should they later be created as outputs by ComfyUI or custom nodes.

The MP4 file format was originally based on the QuickTime File Format (.mov), later evolving to MP4 v2 as MPEG-4, then finally generalized into the ISO base media file format (ISOBMFF). This created a broader standard used as the foundation for multiple container formats, including modern .mov and related types like .m4v (often used for MP4 video within Apple ecosystems). ISOBMFF is a container file format that "defines a general structure for files that contain time-based multimedia data such as video and audio."

To embed metadata from the python side, use the following approach:

        # Create container with movflags option
        container = av.open(temp_path, mode='w', options={'movflags': 'use_metadata_tags'})
        
        # Add metadata
        if prompt is not None:
            container.metadata["prompt"] = json.dumps(prompt)
        
        if extra_pnginfo is not None:
            for x in extra_pnginfo:
                container.metadata[x] = json.dumps(extra_pnginfo[x])

┆Issue is synchronized with this Notion page by Unito

Comment on lines +1320 to +1326
} else if (
file.type === 'video/mp4' ||
file.name?.endsWith('.mp4') ||
file.name?.endsWith('.mov') ||
file.name?.endsWith('.m4v') ||
file.type === 'video/quicktime' ||
file.type === 'video/x-m4v'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In future: refactor this entire section of app.ts that conditionally handles dropped files based on type/extension. It is repeating same logic for every file type and the number of supported output types if increasing

@christian-byrne christian-byrne marked this pull request as ready for review April 26, 2025 19:02
@christian-byrne christian-byrne requested review from a team as code owners April 26, 2025 19:02
@christian-byrne christian-byrne added the New Browser Test Expectations New browser test screenshot should be set by github action label Apr 26, 2025
@christian-byrne christian-byrne merged commit 99cc587 into main Apr 26, 2025
2 checks passed
@christian-byrne christian-byrne deleted the parse-mp4 branch April 26, 2025 19:31
christian-byrne added a commit that referenced this pull request May 20, 2025
Co-authored-by: github-actions <github-actions@github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New Browser Test Expectations New browser test screenshot should be set by github action

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[API Nodes] Support loading workflow from mp4

2 participants