Skip to content

Commit

Permalink
change word
Browse files Browse the repository at this point in the history
  • Loading branch information
Yoongi Kim committed Nov 22, 2018
1 parent 4d30b3d commit ecd683f
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 4 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ usage: python3 auto_crawler.py [--skip true] [--threads 4] [--google true] [--na
--naver NAVER Download from naver.com (boolean)


# Integrity Checking
# Data Imbalance Detection

Detects data imblance based on number of files.

Expand Down
9 changes: 6 additions & 3 deletions auto_crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,10 +147,11 @@ def do_crawling(self):
pool.join()
print('pool join')

self.integrity_check()
self.imbalance_check()

def integrity_check(self):
print('Integrity Checking...')
print('End Program')

def imbalance_check(self):
print('Data imbalance checking...')

dict_num_files = {}
Expand Down Expand Up @@ -190,6 +191,8 @@ def integrity_check(self):
print('Removed {}'.format(dir))

print('Now re-run this program to re-download removed files. (with skip_already_exist=True)')
else:
print('Data imbalance not detected.')


if __name__ == '__main__':
Expand Down

0 comments on commit ecd683f

Please sign in to comment.