Using tasks/keypoint-detection in huggingface.js or transformers.js

[jo-chemla]

Hi there,

I wanted to know if it is currently possible to use keypoint-detection (associated package repo link) (ultimate task is pose estimation) within transformers.js or huggingface.js in a JS app, whether frontend or backend. The safetensor is 50MB, so it seems doable to transfer model weights to client and use the model keypoint detection. I do not see pose estimation task referenced, but probably this can then be tied to custom ransac logic or opencvjs to compute the relative pose from the two images detected keypoints.

I'm especially interested in ETH-CVG/lightglue_disk feature detector and matcher (notebook) and wanted to test it out in-browser without deploying an inference API endpoint.

Would love some pointers if this is already possible. Thanks!