Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ script:
- nosetests $COVER_PACKAGE --with-doctest --with-coverage --with-timer -v --cover-package=$COVER_PACKAGE
- kill $QIITA_PID
- if [ ${TEST_ADD_STUDIES} == "True" ]; then test_data_studies/commands.sh ; fi
- if [ ${TEST_ADD_STUDIES} == "True" ]; then qiita-cron-job ; fi
- if [ ${TEST_ADD_STUDIES} == "True" ]; then qiita-cron-job empty_trash_upload_folder ; qiita-cron-job generate_biom_and_metadata_release ; qiita-cron-job purge_filepaths ; qiita-cron-job purge_files_from_filesystem ; qiita-cron-job update_redis_stats ; fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to pack this into a single script so it doesn't lead to a such a long single line in the configuration file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure ... if you feel strong about it ... let me know.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking about it, this might be useful for other reasons, just added.

- flake8 qiita_* setup.py scripts/*
- qiita pet webserver
addons:
Expand Down
62 changes: 43 additions & 19 deletions scripts/qiita-cron-job
Original file line number Diff line number Diff line change
Expand Up @@ -8,31 +8,55 @@
# The full license is in the file LICENSE, distributed with this software.
# -----------------------------------------------------------------------------

import click

from qiita_db.util import (
purge_filepaths, empty_trash_upload_folder, purge_files_from_filesystem)
purge_filepaths as qiita_purge_filepaths,
empty_trash_upload_folder as qiita_empty_trash_upload_folder,
purge_files_from_filesystem as qiita_purge_files_from_filesystem)
from qiita_db.meta_util import (
update_redis_stats, generate_biom_and_metadata_release)
update_redis_stats as qiita_update_redis_stats,
generate_biom_and_metadata_release as
qiita_generate_biom_and_metadata_release)


# This script will perform these jobs:
# 1. purge_filepaths: remove files from that are leftover in the
# qiita.filepath and are present in the filesystem
# 2. empty_trash_upload_folder: remove files that are present in the trash
# of the upload folders
# 3. update_redis_stats: updates the redis stats information
# 4. generate public releases of biom tables and metadata
#
# Note that is responsability of the Qiita install system admin to add to a
# cron job this script and responsible to define how often it should run
@click.group()
def commands():
pass


@commands.command()
@click.option('--remove/--no-remove', default=True,
help=('remove files from that are leftover in the '
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the help text in here means, would you mind rephrasing. Also, the parenthesis are not needed.

'qiita.filepath and are present in the filesystem'))
def purge_filepaths(remove):
qiita_purge_filepaths(remove)


@commands.command()
@click.option('--remove/--no-remove', default=False,
help='check the filesystem and remove not used')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Files that are not used?

def purge_files_from_filesystem(remove):
qiita_purge_files_from_filesystem(remove)


@commands.command()
@click.option('--remove/--no-remove', default=True,
help=('remove files from the trash folder within the upload '
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for the parenthesis here.

'folders'))
def empty_trash_upload_folder(remove):
qiita_empty_trash_upload_folder(remove)


@commands.command()
def update_redis_stats():
qiita_update_redis_stats()


def main():
purge_filepaths(True)
purge_files_from_filesystem(False)
empty_trash_upload_folder(True)
update_redis_stats()
generate_biom_and_metadata_release('public')
@commands.command()
def generate_biom_and_metadata_release():
qiita_generate_biom_and_metadata_release('public')


if __name__ == "__main__":
main()
commands()