Skip to content

Commit c249212

Browse files
committed
update the script for making hf dataset
1 parent 440e9dd commit c249212

File tree

1 file changed

+1
-3
lines changed

1 file changed

+1
-3
lines changed

examples/script_data_processing/make_hf_dataset.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ def make_json_serializable(input_dict):
1414

1515
return input_dict
1616

17+
1718
def make_hf_dataset(source_dir, target_dir, split='test'):
1819
data_pkl = load_pickle(source_dir)
1920

@@ -56,6 +57,3 @@ def make_hf_dataset(source_dir, target_dir, split='test'):
5657
make_hf_dataset(source_dir=test_data_dir[0], target_dir=test_data_dir[1])
5758
make_hf_dataset(source_dir=dev_data_dir[0], target_dir=dev_data_dir[1], split='dev')
5859
make_hf_dataset(source_dir=train_data_dir[0], target_dir=train_data_dir[1], split='train')
59-
60-
61-

0 commit comments

Comments
 (0)