split qiita cron job #2543
split qiita cron job #2543
Conversation
Codecov Report
@@ Coverage Diff @@
## dev #2543 +/- ##
==========================================
+ Coverage 94.41% 94.41% +<.01%
==========================================
Files 160 160
Lines 19453 19458 +5
==========================================
+ Hits 18367 18372 +5
Misses 1086 1086
Continue to review full report at Codecov.
|
scripts/qiita-cron-job
Outdated
|
|
||
| @commands.command() | ||
| @click.option('--remove/--no-remove', default=True, | ||
| help=('remove files from that are leftover in the ' |
There was a problem hiding this comment.
Not sure what the help text in here means, would you mind rephrasing. Also, the parenthesis are not needed.
scripts/qiita-cron-job
Outdated
|
|
||
| @commands.command() | ||
| @click.option('--remove/--no-remove', default=False, | ||
| help='check the filesystem and remove not used') |
There was a problem hiding this comment.
Files that are not used?
scripts/qiita-cron-job
Outdated
|
|
||
| @commands.command() | ||
| @click.option('--remove/--no-remove', default=True, | ||
| help=('remove files from the trash folder within the upload ' |
There was a problem hiding this comment.
No need for the parenthesis here.
.travis.yml
Outdated
| - kill $QIITA_PID | ||
| - if [ ${TEST_ADD_STUDIES} == "True" ]; then test_data_studies/commands.sh ; fi | ||
| - if [ ${TEST_ADD_STUDIES} == "True" ]; then qiita-cron-job ; fi | ||
| - if [ ${TEST_ADD_STUDIES} == "True" ]; then qiita-cron-job empty_trash_upload_folder ; qiita-cron-job generate_biom_and_metadata_release ; qiita-cron-job purge_filepaths ; qiita-cron-job purge_files_from_filesystem ; qiita-cron-job update_redis_stats ; fi |
There was a problem hiding this comment.
would it make sense to pack this into a single script so it doesn't lead to a such a long single line in the configuration file?
There was a problem hiding this comment.
sure ... if you feel strong about it ... let me know.
There was a problem hiding this comment.
thinking about it, this might be useful for other reasons, just added.
6 participants
Discussing with @jdereus we realized that the full qiita-cron-job is taking too long in the system so I thought splitting in smaller commands will allow us to parallelize the commands and also test each step.