Skip to content

Commit

Permalink
[wip] attempt at limiting concurrent deploys
Browse files Browse the repository at this point in the history
Since fetch_from_genbank can query NCBI up to 5 times for each of the serotypes, try to limit concurrent queries to under 3. Using 2 to be cautious.

Following the format shown at:
nextstrain/ncov#1045
  • Loading branch information
j23414 committed Apr 18, 2023
1 parent 84b7def commit 33c656e
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions ingest/workflow/snakemake_rules/fetch_sequences.smk
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Produces final output as
sequences_ndjson = "data/sequences_{serotype}.ndjson"
"""
workflow.global_resources.setdefault("concurrent_deploys", 2)

def download_serotype(wildcards):
serotype = {
Expand All @@ -24,6 +25,8 @@ def download_serotype(wildcards):
return serotype[wildcards.serotype]

rule fetch_from_genbank:
resources:
concurrent_deploys=1,
output:
genbank_ndjson="data/genbank_{serotype}.ndjson",
params:
Expand Down

0 comments on commit 33c656e

Please sign in to comment.