Skip to content

Latest commit

 

History

History

evaluate

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
__pycache__
__pycache__
 
 
OIE2016_dev.txt
OIE2016_dev.txt
 
 
OIE2016_test.txt
OIE2016_test.txt
 
 
Re-OIE2016-Binary.json
Re-OIE2016-Binary.json
 
 
Re-OIE2016-Portuguese-Binary.json
Re-OIE2016-Portuguese-Binary.json
 
 
Re-OIE2016-Portuguese.json
Re-OIE2016-Portuguese.json
 
 
Re-OIE2016-Spanish-Binary.json
Re-OIE2016-Spanish-Binary.json
 
 
Re-OIE2016-Spanish.json
Re-OIE2016-Spanish.json
 
 
Re-OIE2016.json
Re-OIE2016.json
 
 
argument.py
argument.py
 
 
dev.oie.conll.txt
dev.oie.conll.txt
 
 
evaluate.py
evaluate.py
 
 
extraction.py
extraction.py
 
 
generalReader.py
generalReader.py
 
 
goldReader.py
goldReader.py
 
 
gold_relabel.py
gold_relabel.py
 
 
matcher.py
matcher.py
 
 
oieReader.py
oieReader.py
 
 
readme.md
readme.md
 
 

readme.md

Evaluation

This code is mainly based on the code from the origin OIE2016 repository. The command to run the code is
python evaluate.py [new/old] input_file output_file
"new" means that we use Re-OIE2016 as benchmark and "old" means that we use OIE2016 as benchmark. The input_file is the extraction of openIE system. Each line follows the following format (separated by tab):
sentence confidence_score predicate arg0 arg1 arg2 ...
The script will output the AUC and best F1 score of the system. And the output file is used to draw the pr-curve. The script to draw the pr-curve is here.