Quality-focused Active Adversarial Policy for Safe Grasping in Human-Robot Interaction (QFAAP)

The QFAAP is designed to enhance the safety of vision-guided robot grasping in Human-Robot Interaction (HRI) scenarios. It introduces an Adversarial Quality Patch (AQP) and a Projected Quality Gradient Descent (PQGD) that adapts to human hand shapes from the perspective of benign adversarial attacks, which can be used to reduce the grasping priority of hands and nearby objects, enabling robots to focus on safer, more appropriate grasping targets.

arXiv | Video

If you use this work, please cite:

@inproceedings{clee2025qfaap,
	title={Quality-focused Active Adversarial Policy for Safe Grasping in Human-Robot Interaction},
	author={Chenghao, Li and Razvan, Beuran and Nak Young, Chong},
	booktitle={arXiv:2503.19397},
	year={2025}
}

Contact

Any questions or comments contact Chenghao Li.

Installation

This code was developed with Python 3.8 on Ubuntu 22.04. Python requirements can installed by:

pip install -r requirements.txt

Datasets

Currently, all datasets are supported.

Cornell Grasping Dataset

1. Download and extract the Cornell Dataset.

OCID Grasping Dataset

1. Download and extract the OCID Dataset.

Jacquard Grasping Dataset

1. Download and extract the Jacquard Dataset.

Pre-trained Grasping Models

All pre-trained grasping models for GG-CNN, GG-CNN2, GR-Convnet, and others can be downloaded from here.

Pre-trained AQP

All AQP trained by different grasping models and datasets can be downloaded from here.

Pre-trained Hand Segmentation Models

All pre-trained Hand Segmentation models can be downloaded from here or here.

Training/Evaluation

Training for AQP is done by the AQP_training.py. Training for Grasping model is done by the train_grasping_network.py. And the evaluation process is followed by the training.

Predicting

1. The offline and realtime prediction of QFAAP is done by the QFAAP_offline.py and QFAAP_realtime.py.
2. For the deployment of real-time hand segmentation, please refer to this repository https://github.com/Unibas3D/Upper-Limb-Segmentationp

Running on a Robot

1. Please reference this repository https://github.com/dougsm/ggcnn_kinova_grasping
2. Or https://github.com/clee-jaist/MCIGP