CSV support

[yashpatel6]

• I have read the code review guidelines and the code review best practice on GitHub check-list.

• The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)-[brief_description_of_branch].

• I have set up or verified the branch protection rule following the github standards before opening this pull request.

• I have added my name to the contributors listings in the
  metadata.yaml and the manifest block in the nextflow.config as part of this pull request, am listed
  already, or do not wish to be listed. (This acknowledgement is optional.)

• I have added the changes included in this pull request to the CHANGELOG.md under the next release version or unreleased, and updated the date.

• I have updated the version number in the metadata.yaml and manifest block of the nextflow.config file following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)

• I have tested the pipeline on at least one A-mini sample.

Adding CSV support back in. Inputs can now also be provided with a CSV with empty cells for certain values as needed. For BAM and FASTQ inputs, the CSV remains the same as it previously was. For SRC input, a set of columns are defined. For mix inputs, empty cells can be used, for example to provide BAM and SNV call inputs, rows giving SNV inputs can leave the BAM columns empty.

Regardless of input type, the common columns of: patient, sample, state should always be provided and filled in.

The YAML templates were also updated to be clearer about which keys and values need to be filled in.

Testing Results

CSV input test

Tested with CSV input here: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-csv-support/sample_test

• CSV: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-csv-support/sample_test/mix_input_snv.csv
• Log: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-csv-support/sample_test/meta-csv.log
• Config: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-csv-support/sample_test/template.config

Parsing match test

The parsing function itself was tested for the different input types with corresponding logs to show matching format to YAML: /hot/software/pipeline/metapipeline-DNA/Nextflow/development/unreleased/yashpatel-csv-support/parsing_test/log-{BAM,CRAM,FASTQ,SRC-CNA,SRC-SNV,SRC}.log

[zhuchcn]

This wonderful! I think a lot of people will appreciate this effort to keep supporting CSV input. So we now suport CSV input for FASTQ, BAM, and BAM + SRC inputs, but not FASTQ + SRC inputs (which is totally fine with me)?

[yashpatel6]

This wonderful! I think a lot of people will appreciate this effort to keep supporting CSV input. So we now suport CSV input for FASTQ, BAM, and BAM + SRC inputs, but not FASTQ + SRC inputs (which is totally fine with me)?

Technically, FASTQ + SRC input should also work though I haven't tested it explicitly here

[sorelfitzgibbon]

I like the added explanation for YAML inputs.