Write the paper!

[ArtPoon]

Some notes from my e-mail correspondence with @davidchampredon:

Here's the outline of the manuscript.
Introduction

• Genetic sequences are considered as certain quantities
• But uncertainty is introduced at nearly all the sequencing steps (bio and software)
• So we should quantify uncertainty and propagate it into any genetical analysis
  review of the few studies that dealt with that - not satisfactory

Methods

• probabilistic sequences theoretical framework

• very brief description of sung (full description in another paper)

• Example 1: phylogenetic trees applied to
  a) summary stats on ancestry (tree topology) ? or
  b) clustering? or
  c) source attribution?

• Example 2: ??? (something that is different enough from example 1, no phylogenetic tree)

Results

• results for example 1
• results for example 2

Previously, I think my angle was too focused on the phylogenies reconstruction.

My initial experiment was to:
i) generate a known tree (known sequences and evolution)
ii) simulate uncertainty (observation errors) N times according to the theoretical model using "probabilistic sequences"
iii) infer tree for each of the N simulations
iv) see how all those trees differ using the TN93 distance

I think I shouldn't use TN93 at step iv) because this is, I think, too "abstract". I should probably push the ?analysis further with results that are more intuitive for a broad audience. Examples that come to my mind are the a), b), c) points in Example 1 above... but I would like to pick your brain on this.

Also, I think there should be a second example that is different from example 1 (so, not based on a phylogeny) in order to show the broad applicability of the method. But I'm limited when it comes to choosing something... again you will probably think of something relevant here.

[ArtPoon]

Correspondence on working with SARS-CoV-2 NGS data:

The way the probabilistic sequence is constructed is: for a given position we add up the probabilities of each nucleotide (inferred from the Phred score of each read) and divide by the number of reads for that position. It is basically the mean probability of that nucleotide across all reads. The coverage of that position doesn't affect the probability directly. For example, if the coverage is 10 reads and the probability is 0.9 for each read, it leads to the same probability in the probabilistic sequence as if there would have been 10,000 reads with the same 0.9 probability.

The nucleotide caller of the platforms (Nanopore, Illumina) takes into account the actual value of the coverage in its decision. If coverage < threshold, then the base at that position is ambiguous (even if the, say, 3 reads were all calling A).

So we can have the following situation. Say 5 reads of a position call A with a quality score equivalent to a 1% observation error.
The platform's caller will call an ambiguous N at that position because 5 reads is below its threshold.
The probabilistic sequence will assign a probability of 0.99 to A and 0.0033 to C, G, T. Hence, the entropy is very small.
This, I think, explains the results (that still need to be fully checked) I showed you.

I guess this highlights a problem in the way the probabilistic sequence is constructed: we do not take the coverage value into account (the average is too probably naive). I'm trying to come up with a way to include this, but I feel I'm re-inventing the wheel of the platforms' callers (please let me know if you have relevant papers in mind).

[ArtPoon]

Possible "killer app" examples for paper:

• lineage classification of SARS-CoV-2 genomes with Pangolin - taking sequence uncertainty into account
• quantifying effect of sequence uncertainty on tree reconstruction (@davidchampredon has already examined this to some extent)

[DBecker7]

For Pangolin: Do FASTQ or SAM files for SARS-CoV-2 exist somewhere? Or would I be simulating the possible uncertainties?