📝 EgoMQ Data Preparation

The EgoMQ metadata can be downloaded from the Ego4D official webpage. Follow the annotation conversion step here. Keep the metadata in jsons/ folder. For quickstart, the matadata can be easily downloaded as follows:

wget https://www.cis.jhu.edu/~shraman/EgoVLPv2/datasets/EgoMQ/jsons.tgz
tar -xvzf jsons.tgz && rm jsons.tgz

📊 Results

Method	mAP (%) @ IoU=0.1	mAP (%) @ IoU=0.3	mAP (%) @ IoU=0.5	mAP (avg)
EgoVLPv2 + VSLNet	17.58	11.92	6.90	12.23

⚙️ Pre-extracted EgoMQ Features

Our pre-extracted video features can be downloaded as:

mkdir saved_features 
cd saved_features
wget https://www.cis.jhu.edu/~shraman/EgoVLPv2/pre-extracted_features/EgoMQ/EgoVLPv2.tgz
tar -xvzf EgoVLPv2.tgz && rm EgoVLPv2.tgz

🎯 Fine-tuning on EgoMQ

This script uses PyTorch’s DataParallel (DP) implementation. For feature extraction, please follow these steps. To run head-tuning, modify the Features variable in scripts/train_infer_eval_ego_nce.sh with proper path of extracted features.

# We perform a grid-search for four different hyper-parameters: batch_size, learning_rate, step_size, and step_gamma.
bash scripts/train_infer_eval_ego_nce.sh

The evaluation results will be saved in /outputs/ directory.

# Find the best results
python find_best_parameters.py --root_folder ./outputs/

🙏 Acknowledgement

We use VSGN as task-specific head for EgoMQ.