Workflow Induction Toolkit

We induce interpretable, structured workflows from either human or agent computer-use activities, usually consisting heterogeneous low-level actions. Download apps for and to start!

Read our paper: "How AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations" 🔗

Quick Start 🚀

bash induce_workflow.sh ${DATA_DIR}

Find workflow step descriptions in ${DATA_DIR}/workflow.txt and detailed data in ${DATA_DIR}/workflow.json.

① Installation

git clone https://github.com/zorazrw/workflow-induction-toolkit
cd workflow-induction-toolkit
export OPENAI_API_KEY=${YOUR_API_KEY}

② Record Human Computer-Use Activities

Install the recording tool:

cd computer-recorder
pip install -e .

Enable recording permission on your Mac: go to System Preferences $\rightarrow$ Privacy & Security $\rightarrow$ Accessibility, allow recording for the app that you use to edit the code, e.g., VSCode. Then follow the instructions to configure the system settings.

Run the recording tool by simply running:

crec   # Press `Ctrl+C` to stop the recording.

③ Induce Human Workflows

cd workflow-induction
pip install -r requirements.txt

To parse a human trajectory, we first need to merge duplicate actions:

python get_human_trajectory.py --data_dir <data_dir>

where data_dir should be directory containing the records, e.g., ~/Downloads/ by default.

You can also preprocess agent trajectories into a similar format, get_agent_trajectory.py provides an example for OpenHands agents (run python get_agent_trajectory.py --data_dir <data_dir>).

Then, segment the trajectory based on state transitions:

python segment.py --data_dir <data_dir>

Lastly, perform semantic-based segment merging:

python induce.py --data_dir <data_dir> --auto

where --auto is used to automatically iterate and terminate workflow induction.

The workflow will be saved in the {data_dir}/workflow.json as a JSON file. The high-level step description will be saved in the {data_dir}/workflow.txt as plain texts.

Acknowledgement

Please cite our paper if you find this toolkit useful in your research:

@inproceedings{howai2025wang,
  title = {How AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations},
  author = {Wang, Zora Zhiruo and Shao, Yijia and Shaikh, Omar and Fried, Daniel and Neubig, Graham and Yang, Diyi},
  journal={arXiv preprint arXiv:2510.22780},
  year = {2025},
}

The recording tool is modified based on GUM.