Eyetracking utilization

[smlpt]

Changes made by Ruoshan to implement a bridge for Mastodon tracking to Sciview, as well as utilize eye tracking in Sciview.

[kephale]

Just checking how you're doing on this @smlpt!

It looks pretty merge friendly, but I'm curious if there is anything else you need on the sciview side?

[smlpt]

Just checking how you're doing on this @smlpt!

It looks pretty merge friendly, but I'm curious if there is anything else you need on the sciview side?

I am currently in the process of refactoring many of the tracking classes, because they were all created as command style classes, making it tedious to call them from code (I couldn't really call any member function except for run()). So I'm going to extract their functionality into separate classes, and then the Demo-named classes simply exist for the command purpose (like I already did in 886997a for the EyeTracking class).
I also consolidated most of their functionality into a base class, because several different classes (EyeTrackingDemo, VRControllerTrackingDemo, VRHeadsetTrackingDemo, Test) all shared large chunks of more or less identical code.

So to answer your question: this branch turned into an active construction site again, and I may have to refine a few more things related to the tracking before we can merge it.
Or do you think its better to merge a basic (working) version of the branch first, and then keep working on it in a new branch?

I would also appreciate if this branch could be merged into scenery soon™, in case I need to add other stuff to scenery.

[smlpt]

This should now bring some order into the naming scheme.
NB: The CI is still failing because I am using the "centered" keyword when showing camera messages, and this feature isn't part of a new scenery release yet (but only exists in scenery main, which I include locally).
(Well and also the grid traversal volume sampling from the corresponding scenery branch)

[smlpt]

I cleaned up the files, the PR is now ready for review. The CI fails because we can't use local scenery with it, and there's a scenery function that was updated in main and that didn't make it into a new release yet.
A few classes were mostly copied from Ruoshan and then cleaned up; I didn't have the time yet to test their functionality. I will probably change or remove those classes in the future: VRHeadsetTracking, TrackingTest, VRControllerTracking.

[smlpt]

Any progress on this? @kephale @skalarproduktraum

[smlpt]

@skalarproduktraum @kephale poke poke.

The CI fails because we're missing a scenery version that contains a minor feature (message centering for the Camera class).

[kephale]

@smlpt do you need that updated in main or can you update that in your branch?

[smlpt]

@smlpt do you need that updated in main or can you update that in your branch?

I'm not working on this branch anymore, my latest changes are in the controller-tracking branch which I branched out from this one. But since that branch is also close to completion, I thought it makes sense to merge this one first.

And I think @skalarproduktraum wanted this PR to be merged sometime.

The branch works locally because I use a local scenery for it. My message was just to explain why the CI fails here.

[kephale]

Aha, got it. Well just ping when you want the full review/merge!

[smlpt]

Would appreciate if this PR could be reviewed and merged to main @skalarproduktraum @kephale (whoever finds time first)

[skalarproduktraum]

Hi @smlpt! Happy to review, but in the current state, the CI doesn't run through with compilation errors. Please fix this, and then I'm more than happy to review and merge 🚢

[kephale]

Also happy to review and see this merged! If you ping when the CI is working I'll try to check it out asap

[smlpt]

@skalarproduktraum we will need a new scenery release then to fix the CI (see above comments)

[smlpt]

There are a few classes from Ruoshans original project that made their way into the PR and can maybe also be removed, but I'm not sure. At least they are not used for the current manvr3d features. These include:

• OpenTrackFile
• NodeTaggedEvent
• EyeTrackingCommand (original purpose was as standalone sciview usage, but now EyeTracking relies on callbacks to Mastodon)
• TrackingDragBehavior (looks like it was never implemented by Ruoshan)
• TrackingTest (legacy code for reading a CSV and building tracks in sciview. Not connected to Mastodon)
• TrackingTestCommand
• VRControllerTracking (uses the controller direction instead of the gaze direction to track cells. Not updated to work with Mastodon because we use a different kind of controller tracking)
• VRControllerTrackingCommand
• VRHeadsetTracking (I see no real use for this over eye tracking. head movement instead of eye movement is probably even more confusing to use for tracking)
• VRHeadsetTrackingCommand