Research Update Meeting 10/02/2025

[SimoneMic]

Update meeting about the current work in progres

Chair:

• @SimoneMic

Attendees:

• @steb6
• @andrearosasco
• @PasMarra
• @vigisushrutha23

Absent:

• @enyen
• @lornat75

---

@andrearosasco

TODAY

• Improved Kernel Density method and working on testing it on other tasks. The results are good, and if confirmed will be added to the paper and submitted.

OLD MEETING

• Paper: improvement on the Diffusion Policy on how to better handle rotations. (keypoint-based filtering gave a really slight improvement, almost not worth it, but more complex. Testing Kernel Density Estimation)
• Fixed teleoperation code on the Panda robot (done for the master student)

@steb6

TODAY

• Update on MetaCub: Ste will work on metaCub after paper deadline (we don't trust new Fellow skills)
• Still working on AR for Paper Submission.

OLD MEETING

• MetaCub in stand-by until paper is finished. Probably a new fellow will start converting the code from python to c++ and implementing Meta API for oculus image streaming.
• Action Recognition: Paper Abstract done (for ECCV). Currently collecting data for the paper: testing different backbones with mixed open and closed set methods for few shot. (some code improvements done and replicated the performances of some papers).

@vigisushrutha23

TODAY

• Finishing updating on fixes on MPC
• Setting up meeting with Mohammed and @SimoneMic for paper submission

OLD OLDMEETING

• ergoCub on Mujoco: figured out where the bug was in the port data formats. But the implementation of the remote-motorcontrolboard-remapper needs to implement some further methods, which are missing in python. Vig will ask Silvio a hand on this and how to figure out the mujoco-yarp integration.
• Probably there is a bug on bimanual which is mixing the encoder values with the references. This could generate the drift error. This issue will be investigated.
• After the mujoco part is done, will work on MPC controller implementation in ergoCub.

@PasMarra

TODAY

• Xela: Collecting data for 5 and 7 taxels and analyzing how this will impact force estimation
• ergoCub Hand: working with Genesio to create couplings devices for hand joint coupling on the fingers, these will stream the joint positions on yarp ports. https://github.com/icub-tech-iit/ergocub-software/tree/feat/couplingYARPNWSHands
• grasping-baselines: issue on scoring the grasp candidates. Brand is testing the evaluation function. Why the best grasp is not picked?

OLD MEETING

• Xela: Collected data and trained simple three layers regression models. However each model has a great error. This is hypothesized to be due to the fact that each sensors does an internal calibration at startup, changing it's internal characteristics. Further investigating this issue, but if confirmed, the sensors could be useless.
• ErgoCub Hand: Working on joint coupling on the fingers. The goal is to achieve a Force PID on the fingers using Xelas as feedback. Managed to figure out how to map the encoder reading on the motor to each of the not-measured joints on the finger (

@enyen

ABSENT
TODAY

• Achieved first result with using a single policy for all manipulators, using DPPO and the new anchor points refinements (for consistent ref frame).
• Working on adding depth and vision for training the policy.

OLD MEETING

• Implemented and trained the automatic hand frame placement (Iterative Palm Frame Alignment) on different manipulators. In this way we have a consistently oriented frame for each end effector.
• Fine-tuning the diffusion policy on PChands with this new frame alignement in place.
• Converting a new online open dataset into a format that could be used for PChands.

@SimoneMic

TODAY

• VLN: added LLM agent for texting and image reasoning: the issue is that image to text encoding takes too many tokens, and the reasoning from ChatGPT is not that strong. Need to further investigate accuracy and if it's worth it.
• ergoCub: modified nav2 theta-star planner for having proper plan pose orientation (i.e. each pose of the path points towards the next one, instead of having the 0.0 rotation).
• Started developing a docker for ros2 stuff on the robot torso. The main issue is aligning the robotology-superbuilds. If Tech does some changes, these need to be ported over each time. This could become very chaotic and hard to manage efficiently.

OLD MEETING

• Finalized Visual-Language-Navigation code with ChatGPT-4o. The information about the objects in the environment is expressed in the robot frame.
• Improved raycasting: now all the points of the camera pointcloud are being considered.
• Now the agent can also handle orientation and encumbrance of the robot.