This repository provides an annotated dataset of papers accepted as long papers at ACL from 2016 to 2023. The papers targeted for annotation are limited to those where the research question in the paper is in the format of "Can a certain 'problem' be solved by a certain 'method'?"
The papers included in the dataset are selected from long papers accepted at ACL between 2016 and 2023. These papers propose new methods for specific tasks and verify their effectiveness.
The dataset contains the following information:
- Paper title
- Paper citation
- Paper abstract
- Paper introduction
- Research Question (RQ) generated by GPT-4 from the paper's abstract and introduction
- A score evaluating whether the RQ generated by GPT-4 can estimate the true problem on a 3-point scale from 0 to 2
- A score evaluating whether the RQ generated by GPT-4 can estimate the true method on a 3-point scale from 0 to 2
- A score evaluating whether the RQ generated by GPT-4 follows the specific format: "Can a certain 'problem' be solved by a certain 'method'?" on a 2-point scale of 0 or 1