v0.5 Roadmap

[j20001970]

Framework

• Try out mediapipe::api2 (Introduce MediaPipeGraphBuilder and MediaPipeGraphNode #57 )

Vision Tasks

• HolisticLandmarker (Add holistic landmarker graph #58)

🐧 Linux

• Pipewire backend for CameraHelper
• Build with Freedesktop SDK (Flatpak) in CI

🍎 macOS

• macOS CPU build
• CameraHelper backend using either OpenCV or AVFoundation (maybe share code with iOS?)

🌐 Web

• Web CPU build (GDMP for web #23) (done but most tasks still not work)

Multiple Architectures

• Android x86_64
• Linux arm64 Postponed due to XNNPACK issues on GCC14
• Windows arm64 Postponed
• macOS arm64

OpenCV

• Avoid linking entire libraries to reduce size

Move to CameraServer-based approach

• [Upstream] Enable custom CameraFeed in GDExtension (Add activate_feed and deactivate_feed virtual bind to CameraFeed godotengine/godot#97571)
• Deprecate MediaPipeCameraHelper

[GeorgeS2019]

@j20001970
@Malcolmnixon
Is this project helpful for the HolisticLandmarker works?
https://github.com/purgeme/mediapipe_cpp_lib/blob/master/src/mcl_basic_holistic.cc

[GeorgeS2019]

Is holistic Landmark next after we have v0.10.15 https://github.com/j20001970/GDMP/actions/runs/10896000612

[j20001970]

Is holistic Landmark next after we have v0.10.15 https://github.com/j20001970/GDMP/actions/runs/10896000612

As of v0.10.15, there is still no C++ API like other vision tasks for HolisticLandmarker, only graph class HolisticLandmarkerGraph is defined in holistic_landmarker_graph.cc. While I do not know if official mediapipe team has plan to implement it, I would create a HolisticLandmarker graph usage demo on GDMP-demo as the completion of all available vision-related use cases if there are still no official task API before the next GDMP release.

[GeorgeS2019]

we need to eventually a way from motion capture using godot4 mediapipe to feed directly captured motion to Godot 4 MotionMatching

[GeorgeS2019]

Any update to HolisticLandmarker with the latest update?

[j20001970]

Any update to HolisticLandmarker with the latest update?

MediaPipe v0.10.18 has XNNPACK compilation issue on Windows apparently, so we don't update mediapipe submodule for now, but that will not prevent the progress of bringing up HolisticLandmarker task.

Currently MediaPipeGraphBuilder and MediaPipeGraphNode is being worked on, using MediaPipe's api2 API, might publish the development branch in a few days. HolisticLandmarker graph will be created and configured using these classes, as demonstrated in MediaPipe's test code. There might be other classes that need to polish up before these two classes becomes usable.

[j20001970]

Closed with a8874b8