Repository navigation
Conversation
Adds the MediaPipe Face/Hand/Pose Detection + Landmark and Selfie Segmentation nodes, backed by the MPSGraph-based MPS-MediaPipe package (extracted so it's usable standalone), plus the Joint Bilateral Filter and MPS Guided Filter nodes selfie segmentation's mask refinement uses. Via Claude Sonnet 5
…ause why would this be fucking easy?
vade
force-pushed
the
feature/media-pipe-mps
branch
from
September 18, 2026 22:10
b9173f9 to
8508f75
Compare
…icient encoding/submission/memory usage for larger MPS models.
…lfie Segmentation, and rework Guided/Joint Bilateral filtering - Replace each MediaPipe node's own static model cache + lock with a single process-wide SharedModelCache, so node instances sharing a detector/landmark model share one compile and enough in-flight slots instead of starving each other. - Rework MediaPipeSelfieSegmentationNode to stay entirely on the GPU: the projection kernel now reads the model's output buffer directly, removing the CPU round-trip through lastMaskLogits. - Rework MPSGuidedFilterNode's low-resolution regression stage to use a new CanonicalResample kernel so the guide is resampled through its own texture transform to the signal's resolution before filtering. - Fix and extend JointBilateralFilterNode/JointBilateralFilter.metal and their tests. - Add mps-architecture-spec.html documenting the MPS SPM package command-buffer ownership design. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6bPVeUMgqekBD8TQ4uRsW
Converts the MPS execution architecture doc to Markdown and updates it to reflect the command-buffer ownership design actually implemented (GraphRenderer.preDraw() owns the one shared MPSCommandBuffer; MPS SPM packages take MPSCommandBuffer directly and never wrap or commit it), superseding the earlier commit: Bool-inside-the-package approach the HTML doc described. Referenced from ARCHITECTURE.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6bPVeUMgqekBD8TQ4uRsW
Adds EfficientTAMTrackerNode (video object tracking from a point prompt, forward-only, single object; 128x128 mask + region, centroid, presence; MPSGraph), backed by the local MPS-EfficientTAM package, and registers it with FabricCoreNodesPlugin. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6bPVeUMgqekBD8TQ4uRsW
Adds the MediaPipe Face/Hand/Pose Detection + Landmark and Selfie Segmentation nodes, backed by the MPSGraph-based MPS-MediaPipe package (extracted so it's usable standalone), plus the Joint Bilateral Filter and MPS Guided Filter nodes selfie segmentation's mask refinement uses. Via Claude Sonnet 5
…ause why would this be fucking easy?
…icient encoding/submission/memory usage for larger MPS models.
…lfie Segmentation, and rework Guided/Joint Bilateral filtering - Replace each MediaPipe node's own static model cache + lock with a single process-wide SharedModelCache, so node instances sharing a detector/landmark model share one compile and enough in-flight slots instead of starving each other. - Rework MediaPipeSelfieSegmentationNode to stay entirely on the GPU: the projection kernel now reads the model's output buffer directly, removing the CPU round-trip through lastMaskLogits. - Rework MPSGuidedFilterNode's low-resolution regression stage to use a new CanonicalResample kernel so the guide is resampled through its own texture transform to the signal's resolution before filtering. - Fix and extend JointBilateralFilterNode/JointBilateralFilter.metal and their tests. - Add mps-architecture-spec.html documenting the MPS SPM package command-buffer ownership design. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6bPVeUMgqekBD8TQ4uRsW
Converts the MPS execution architecture doc to Markdown and updates it to reflect the command-buffer ownership design actually implemented (GraphRenderer.preDraw() owns the one shared MPSCommandBuffer; MPS SPM packages take MPSCommandBuffer directly and never wrap or commit it), superseding the earlier commit: Bool-inside-the-package approach the HTML doc described. Referenced from ARCHITECTURE.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6bPVeUMgqekBD8TQ4uRsW
vade
force-pushed
the
feature/media-pipe-mps
branch
from
September 21, 2026 20:29
bea2094 to
2fa55a0
Compare
# Conflicts: # Fabric/Nodes/Plugin/FabricCoreNodesPlugin.swift # NODES.md # Package.swift
… the release gets triggered from.
…+ ane choice, setiing panel updates, async/sync choice, fixes and port clean up
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the MediaPipe Face/Hand/Pose Detection + Landmark and Selfie Segmentation nodes, backed by the MPSGraph-based MPS-MediaPipe package (extracted so it's usable standalone), plus the Joint Bilateral Filter and MPS Guided Filter nodes selfie segmentation's mask refinement uses.