Skip to content

Commit

Permalink
entrypoint-aws-batch: Upload .snakemake/metadata/ too
Browse files Browse the repository at this point in the history
Snakemake stores state information per input/output here and uses it to
determine if it needs to re-run rules or not.  It seems akin to the file
mtimes which we already take care to preserve on upload/download.
Additionally, the metadata recorded is used in Snakemake's report
generation and is generally useful for looking at workflow statistics.

Continue to not upload all of .snakemake/ en masse because it can
potentially contain files that interfere with local usage and/or are
large and unnecessary.

Resolves: <nextstrain/cli#373>
Related-to: <nextstrain/cli#374>
  • Loading branch information
tsibley committed Jun 17, 2024
1 parent ccac078 commit 442a5b7
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions entrypoint-aws-batch
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ case "$NEXTSTRAIN_AWS_BATCH_WORKDIR_URL" in
return "$zipstatus"
fi
}
zip -u "$PWD.zip" -r . --exclude ".snakemake/*" || succeed-when-nothing-to-update $?
zip -u "$PWD.zip" -r . --include ".snakemake/log/*" || succeed-when-nothing-to-update $?
zip -u "$PWD.zip" -r . --exclude ".snakemake/*" || succeed-when-nothing-to-update $?
zip -u "$PWD.zip" -r . --include ".snakemake/log/*" ".snakemake/metadata/*" || succeed-when-nothing-to-update $?
aws s3 cp --no-progress "$PWD.zip" "$NEXTSTRAIN_AWS_BATCH_WORKDIR_URL"
;;
s3://*)
Expand Down

0 comments on commit 442a5b7

Please sign in to comment.