Replace fetch-from-biosample with NCBI Dataset CLI

[joverlee521]

Description of proposed changes

Originally I was going to update fetch-from-biosample to use the v2 NCBI Dataset API as described in #477. However, I remembered #420 and realized that we can remove the script entirely if we just download the BioSample report as part of the NCBI Dataset CLI command.

Related issue(s)

Resolves #477, #420

Checklist

• Checks pass
• Trial run

[joverlee521]

Trial run successfully completed!

Compared the uploaded biosample.tsv.zst from the trial run with the production run and found no difference 🎉

1. Download files from S3, specifically used older version from prod that was uploaded the same day as the trial run.

$ aws s3 cp s3://nextstrain-data/files/ncov/open/trial/update-biosample/biosample.tsv.zst data/new-biosample.tsv.zst
$ zstd -T0 -dcq data/new-biosample.tsv.zst > data/new-biosample.tsv
$ aws s3api get-object --bucket nextstrain-data --key files/ncov/open/biosample.tsv.zst --version-id HTrEUbLJK2Qpauj_qv11Ehrv3j40Zf3o data/old-biosample.tsv.zst
$ zstd -T0 -dcq data/old-biosample.tsv.zst > data/old-biosample.tsv

2. Sort records by BioSample accession

$ csvtk sort -t -k biosample_accession data/new-biosample.tsv > data/sorted-new-biosample.tsv
$ csvtk sort -t -k biosample_accession data/old-biosample.tsv > data/sorted-old-biosample.tsv

3. Diff files

diff data/sorted-old-biosample.tsv data/sorted-new-biosample.tsv > /tmp/diff

No difference between the files 🎉

[joverlee521]

I plan to merge this tomorrow morning if there's no feedback.