Skip to content

Commit b3a527a

Browse files
Ruiqi ZhongRuiqi Zhong
authored andcommitted
adding issues to README
1 parent 786e753 commit b3a527a

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ Include `--progress_bar_for_each_datapoint` if you suspect that the execution go
7676

7777
## Evaluation for Other Classical Text-to-SQL Datasets
7878

79+
**NOTE** Thanks to @rizar we identified several issues with the current release. Please use it only for the purpose of preliminary exploration.
80+
7981
The prior work on classical text-to-sql datasets (ATIS, Academic, Advising, Geography, IMDB, Restaurants, Scholar, Yelp) usually reports the exact string match accuracy and execution accuracy over a single database content, which either exaggerates or deflates the real semantic accuracy.
8082

8183
The test set for classical text-to-sql datasets are adopted from [this repo](https://github.com/jkkummerfeld/text2sql-data). We used all the test splits if the test split is defined, and the entire dataset otherwise. We also rewrite the SQLs to conform with the style in the Spider dataset.

0 commit comments

Comments
 (0)