mechanism to merge shotgun features after atropos but before shogun

[FranckLejzerowicz]

To ease EBI submission and make it at the actual experimental level, we may submit shotgun data to Qiita per lane (i.e. one prep per sequencing run and lane in order to have submitted the data as it comes out of the sequencing instrument).
Yet, for some projects, high/desired sequencing depth might only be achieved by running several run/lanes. Hence, one would have to uploaded >1 prep (i.e. >1 sequenced sample; >1 fastq files pair) per biological sample.
The problem is that for now and as far as know, it is only at the biom table level that sequences can be merged in Qiita (through the Analysis functionalities, isn't it?), whereas to harness the full power of Shogun, it is much better to pass it all the reads of a biological of interest, i.e. the replicates that may be scattered in various preps (thanks @qiyunzhu for the insight here!).
Therefore, a mechanism to merge features amongst Qiita preps after Atropos (or at any step during the shotgun workflow) may be very useful.
Thanks! F

[qiyunzhu]

@antgonza and I just discussed. We believe that:

1. The BIOM tables generated by Bowtie2 are completely mergeable (i.e., the merged table is identical to the table generated from merged data).

2. The BIOM tables generated by BURST under the "capitalist" mode are not perfectly mergeable. This is because "capitalist" optimizes the resulting profile by minimizing the number of species reported when there are ambiguities. This design is thought to be able to reduce false positives. Although the difference between a merged table and a table from merged data is not significant.

3. The BIOM table generated by BURST without the "capitalist" mode are perfectly mergeable.

[antgonza]

Thanks @qiyunzhu.

@FranckLejzerowicz, summarizing: Qiita relies on options that generate the same answer if the raw sequences were processed together or separately and we plan to keep doing that for the future AKA no need to merge raw sequences, let's just use the resulting BIOMs.