You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Bugs in Data Preprocess and may cause the wrong evaluation results: when there are multiple events in a sentence, the output data only contains one event and misses other events.
#22
Open
Lee-zix opened this issue
Aug 10, 2023
· 0 comments
Nice work and thanks for publishing your codes and datasets! I find a problem in your data preprocess that may cause the wrong evaluation results: when there are multiple events in a sentence, the output data only contains one event and misses other events.
For example, in the file "EE/ACE05/text.json", the parsed results of your codes are as follows:
{
"sentence": "Welch also wants details on Jane Beasley Welch ' s salary , benefits , retirement plan and other compensation paid to her .",
"events": [
{
"trigger": "retirement",
"type": "end position",
"pos": [],
"arguments": [
{
"name": "Jane Beasley Welch",
"role": "person",
"pos": []
}
]
}
]
}
However, in the ACE05 datasets, there are two events in the sentence. The right parsing results of the sentence are as follows:
{
"event_mentions": [
{
"event_type": "Personnel:End-Position",
"id": "APW_ENG_20030325.0786-10-EV0",
"trigger": {
"start": 13,
"end": 14,
"text": "retirement"
},
"arguments": [
{
"entity_id": "APW_ENG_20030325.0786-10-E1",
"text": "Jane Beasley Welch",
"role": "Person"
}
]
},
{
"event_type": "Transaction:Transfer-Money",
"id": "APW_ENG_20030325.0786-10-EV1",
"trigger": {
"start": 18,
"end": 19,
"text": "paid"
},
"arguments": [
{
"entity_id": "APW_ENG_20030325.0786-10-E",
"text": "Jane Beasley Welch",
"role": "Recipient"
}
]
}
]
}
If your files are not the same as the origin datasets, the evaluation results in your paper may not be directly compared to the baselines. Please check this problem. Thanks very much.
The text was updated successfully, but these errors were encountered:
Nice work and thanks for publishing your codes and datasets! I find a problem in your data preprocess that may cause the wrong evaluation results: when there are multiple events in a sentence, the output data only contains one event and misses other events.
For example, in the file "EE/ACE05/text.json", the parsed results of your codes are as follows:
{
"sentence": "Welch also wants details on Jane Beasley Welch ' s salary , benefits , retirement plan and other compensation paid to her .",
"events": [
{
"trigger": "retirement",
"type": "end position",
"pos": [],
"arguments": [
{
"name": "Jane Beasley Welch",
"role": "person",
"pos": []
}
]
}
]
}
However, in the ACE05 datasets, there are two events in the sentence. The right parsing results of the sentence are as follows:
{
"event_mentions": [
{
"event_type": "Personnel:End-Position",
"id": "APW_ENG_20030325.0786-10-EV0",
"trigger": {
"start": 13,
"end": 14,
"text": "retirement"
},
"arguments": [
{
"entity_id": "APW_ENG_20030325.0786-10-E1",
"text": "Jane Beasley Welch",
"role": "Person"
}
]
},
{
"event_type": "Transaction:Transfer-Money",
"id": "APW_ENG_20030325.0786-10-EV1",
"trigger": {
"start": 18,
"end": 19,
"text": "paid"
},
"arguments": [
{
"entity_id": "APW_ENG_20030325.0786-10-E",
"text": "Jane Beasley Welch",
"role": "Recipient"
}
]
}
]
}
If your files are not the same as the origin datasets, the evaluation results in your paper may not be directly compared to the baselines. Please check this problem. Thanks very much.
The text was updated successfully, but these errors were encountered: