Feature/bwamem2 #54
Feature/bwamem2 #54
Conversation
| @@ -0,0 +1,272 @@ | |||
| #!/usr/bin/perl | |||
There was a problem hiding this comment.
Already reviewed by @AndyMenzies
| @@ -45,9 +45,9 @@ cd $SETUP_DIR | |||
| ## biobambam2 first | |||
| BB_INST=$INST_PATH/biobambam2 | |||
| if [ ! -e $SETUP_DIR/bbb2.sucess ]; then | |||
| curl -sSL --retry 10 https://github.com/gt1/biobambam2/releases/download/${VER_BBB2}/biobambam2-${VER_BBB2}-x86_64-etch-linux-gnu.tar.gz > distro.tar.gz | |||
| curl -sSL --retry 10 $BBB2_URL > distro.tar.gz | |||
There was a problem hiding this comment.
Updates to current biobambam2
| @@ -67,6 +67,18 @@ curl -sSL https://cpanmin.us/ > $SETUP_DIR/cpanm | |||
| perl $SETUP_DIR/cpanm --no-wget --no-interactive --notest --mirror http://cpan.metacpan.org -l $INST_PATH App::cpanminus | |||
| rm -f $SETUP_DIR/cpanm | |||
|
|
|||
| ## scramble (from staden) | |||
There was a problem hiding this comment.
Biobambam2 no longer includes scramble, don't loose as downstream dependencies
| const my $BAMBAM_DUP => q{%s level=0 %s | %s tmpfile=%s level=0 markthreads=%d M=%s.met %s| pee '%s tmpfile=%s index=1 md5=1 numthreads=%d md5filename=%s.md5 indexfilename=%s.%s > %s' '%s -o %s.bas -@ %d'}; | ||
| const my $BAMBAM_MERGE => q{%s %s tmpfile=%s level=0 %s| pee '%s tmpfile=%s index=1 md5=1 numthreads=%d md5filename=%s.md5 indexfilename=%s.%s > %s' '%s -o %s.bas -@ %d'}; | ||
| const my $BAMBAM_DUP_CRAM => q{%s level=0 %s | %s tmpfile=%s M=%s.met markthreads=%s level=0 %s| %s -r %s -t %d -I bam -O cram %s | tee %s | %s index - %s.crai}; | ||
| const my $BAMBAM_MERGE_CRAM => q{%s %s tmpfile=%s level=0 %s| %s -r %s -t %d -I bam -O cram %s | tee %s | %s index - %s.crai}; | ||
|
|
||
| const my $LANE_BAMBAM_MERGE => q{%s %s tmpfile=%s level=0 | pee '%s tmpfile=%s index=1 md5=1 numthreads=%d md5filename=%s.md5 indexfilename=%s.%s > %s' '%s -o %s.bas -@ %d'}; | ||
| const my $LANE_BAMBAM_MERGE_CRAM => q{%s %s tmpfile=%s level=0 | %s -r %s -t %d -I bam -O cram %s | tee %s | %s index - %s.crai}; | ||
| const my $LANE_BAMBAM_DUP => q{%s level=0 %s | %s -l 0 -m | %s tmpfile=%s level=0 markthreads=%d M=%s.met | %s -l 0 -p | pee '%s tmpfile=%s index=1 md5=1 numthreads=%d md5filename=%s.md5 indexfilename=%s.%s > %s' '%s -o %s.bas -@ %d'}; | ||
| const my $LANE_BAMBAM_DUP_CRAM => q{%s level=0 %s | %s -l 0 -m | %s tmpfile=%s level=0 markthreads=%d M=%s.met | %s -l 0 -p | %s -r %s -t %d -I bam -O cram %s | tee %s | %s index - %s.crai}; | ||
|
|
There was a problem hiding this comment.
Switching tee for pee. This means that bam_stats can work from the decompressed stream.
| @@ -97,6 +103,117 @@ sub bam_to_grouped_bam { | |||
| return PCAP::Threaded::touch_success(File::Spec->catdir($tmp, 'progress'), $index); | |||
| } | |||
|
|
|||
| sub merge_or_mark_lanes { | |||
There was a problem hiding this comment.
Already reviewed by @AndyMenzies
| @@ -56,6 +56,17 @@ sub bwa_mem_max_cores { | |||
| return $BWA_MEM_MAX_CORES; | |||
| } | |||
|
|
|||
| sub bwamem2_version { | |||
There was a problem hiding this comment.
bwa-mem2 has a much cleaner version string... root command is different too
| @@ -83,7 +94,7 @@ sub mem_setup { | |||
| } | |||
| # do some checking to ensure input BAM/CRAM hasn't been through mismatchQc | |||
| # if it has check for use of at least bammaskflags | |||
| PCAP::Bam::mismatchQc_checks($options->{'raw_files'}); | |||
| PCAP::Bam::mismatchQc_checks($options->{'raw_files'}) unless($skip_mmqc_check); | |||
There was a problem hiding this comment.
If merging pre-generated lanes we don't want to block stuff data processed by mmQc. This step is reused by mapping and merging, hence the distinction
| warn "\nGeneral output can be found in this file: $out\n"; | ||
| warn "Errors can be found in this file: $err\n\n"; | ||
| die "Wrapper script message:\n".$_; | ||
| warn "\nTHREAD_EXITED: General output can be found in this file, last 10 lines below: $out\n"; |
There was a problem hiding this comment.
As discussed wit @drjsanger on Slack, makes viewing of logs in wrapping schedulers etc easier.
| @@ -32,6 +32,7 @@ subtest 'Initialisation checks' => sub { | |||
|
|
|||
| subtest 'Non object checks' => sub { | |||
| ok(PCAP::Bwa::bwa_version(), 'Version returned for BWA'); | |||
| ok(PCAP::Bwa::bwamem2_version(), 'Version returned for bwa-mem2'); | |||
There was a problem hiding this comment.
Confirm version check works
|
Everything has been tested under CI. I've commented though-out the changes area so it's clear what each change relates to and if previously reviewed. @drjsanger or @AndyMenzies please can you review this? |
|
mergeOrMark CI branch has just the addition of the new script and the extra CI elements. bwamem2 CI branch has bwamem2 and the tee->pee changes. |
|
The Docker install path is using build/opt-build.sh for installation, which contains both bwamem and bwamem2. However setup.sh only installs bwa, not bwa2, and the tests need both. Might cause issues. Other than that it looks good. |
@AndyMenzies I think you've missed the critical difference between the blocks. Line 115 in 7b6f57a FYI, it's much easier to track if something is resolved if you raise a comment on the diff as it can actually be marked as a query and then resolved. |
|
... and I missed the critical item in your message, as you said, |
…to Dockerfile for other methods
Merge and mark items have been approved in #50