PICO on Hybrid mode will zero any mouth blendshape while speaking

[miranda1000]

Old PICO connect APP allows the user to choose if they want "Hybrid" facetracking, or "face-only":

On new apps this defaults to "Hybrid" and cannot be changed using UI.

Hybrid facetracking cause to listen to the microphone, and if it detect sound that overrides the standard ft parameters. For "face-only" the values are all set to 0: PICOLogs-face_only.csv

Seeing the logs on hybrid we should ignore (keep the last) face blendshapes if any of the audio paramerters are >0.001
PICOLogs.csv

Regarding the eye tracking, it seems that we can trust those params.

Note: both logs I did the same testing, that consist in opening the mouth, keep saying "aaa" while blinking, stop making sound, and finally closing the mouth.

[miranda1000]

Discord thread reporting this bug: https://discord.com/channels/849300336128032789/1347913796009590855

[miranda1000]

development will be done on https://github.com/regzo2/PicoStreamingAssistantFTUDP/tree/fix/hybrid-ft

[miranda1000]

After a second look at the captured ft data, you can notice how the mouth_open blendshape goes down before the sound gets reported.

Although fixing this is possible (using AI or a small delay before reporting the data), I believe the better approach is to just change PICO Connect to "face-only".