Issue Reading 10X GEX+ATAC Multiome Data

[guptas2005]

Hello! I am running into an issue with using SCALPEL on 10X multiome data, specifically 10X GEX+ATAC scRNA seq.

This is my input command:

nextflow run -resume /pathto/SCALPEL/main.nf \
--sequencing chromium \
--samplesheet /pathto/samplesheet.csv \
--transcriptome /pathto/Mus_musculus.GRCm38.cdna.all.fa \
--gtf /pathto/Mus_musculus.GRCm38.98.gtf \
--ipdb /pathto/mm10.polyA.track \
--clusters /pathto/cell_cluster_mapping.tsv \
--outputDir /pathto/scalpel_output \

This is the output I am receiving. I would appreciate any guidance. I'm wondering if this is an issue with the 10X Cellranger output due to Multiome data inherently? Or is there a mistake on my end? Thank you in advance:

executor >  local (2)
[-        ] annotation_preprocessing:salmon_transcriptome_indexing                | 0 of 1
[-        ] annotation_preprocessing:salmon_bulk_quantification                   -
[-        ] annotation_preprocessing:tpm_counts_average                           -
[-        ] annotation_preprocessing:isoform_selection_weighting                  -
[3e/9ec145] rea…g:samples_loading:read_10x (8_1219_F_Multiome, 8_1219_F_Multiome) | 0 of 16
[-        ] reads_processing:samples_loading:bam_splitting                        -
[-        ] reads_processing:bedfile_conversion                                   -
[-        ] reads_processing:reads_mapping_and_filtering                          -
[-        ] reads_processing:ip_splitting                                         -
[-        ] reads_processing:ip_filtering                                         -
[-        ] isoform_quantification:probability_distribution                       -
[-        ] isoform_quantification:fragment_probabilities                         -
[-        ] isoform_quantification:cells_splitting                                -
[-        ] isoform_quantification:em_algorithm                                   -
[-        ] isoform_quantification:cells_merging                                  -
[-        ] isoform_quantification:dge_generation                                 -
[-        ] apa_characterization:differential_isoform_usage                       -
[-        ] apa_characterization:generation_filtered_bams                         -
ERROR ~ Error executing process > 'reads_processing:samples_loading:read_10x (1135_F_Multiome, 1135_F_Multiome)'

Caused by:
  Process `reads_processing:samples_loading:read_10x (1135_F_Multiome, 1135_F_Multiome)` terminated with an error exit status (1)


Command executed:

  Rscript /mnt/isilon/cristancho_data/Shikhar/Splicing/IsoformSwitchAnalyzeR/SCALPEL/SCALPEL/src/read_10X.R 1135_F_Multiome
  ln -s 1135_F_Multiome/outs/possorted_genome_bam.bam 1135_F_Multiome.bam
  ln -s 1135_F_Multiome/outs/possorted_genome_bam.bam.bai 1135_F_Multiome.bam.bai
  gunzip -c 1135_F_Multiome/outs/filtered_feature_bc_matrix/barcodes.tsv.gz > 1135_F_Multiome.barcodes

Command exit status:
  1

Command output:
  [1] "reading file..."

Command error:
  
  Attaching package: ‘dplyr’
  
  The following objects are masked from ‘package:data.table’:
  
      between, first, last
  
  The following objects are masked from ‘package:stats’:
  
executor >  local (2)
[-        ] annotation_preprocessing:salmon_transcriptome_indexing                | 0 of 1
[-        ] annotation_preprocessing:salmon_bulk_quantification                   -
[-        ] annotation_preprocessing:tpm_counts_average                           -
[-        ] annotation_preprocessing:isoform_selection_weighting                  -
[3e/9ec145] rea…g:samples_loading:read_10x (8_1219_F_Multiome, 8_1219_F_Multiome) | 1 of 16, failed: 1
[-        ] reads_processing:samples_loading:bam_splitting                        -
[-        ] reads_processing:bedfile_conversion                                   -
[-        ] reads_processing:reads_mapping_and_filtering                          -
[-        ] reads_processing:ip_splitting                                         -
[-        ] reads_processing:ip_filtering                                         -
[-        ] isoform_quantification:probability_distribution                       -
[-        ] isoform_quantification:fragment_probabilities                         -
[-        ] isoform_quantification:cells_splitting                                -
[-        ] isoform_quantification:em_algorithm                                   -
[-        ] isoform_quantification:cells_merging                                  -
[-        ] isoform_quantification:dge_generation                                 -
[-        ] apa_characterization:differential_isoform_usage                       -
[-        ] apa_characterization:generation_filtered_bams                         -
ERROR ~ Error executing process > 'reads_processing:samples_loading:read_10x (1135_F_Multiome, 1135_F_Multiome)'

Caused by:
  Process `reads_processing:samples_loading:read_10x (1135_F_Multiome, 1135_F_Multiome)` terminated with an error exit status (1)


Command executed:

  Rscript /mnt/isilon/cristancho_data/Shikhar/Splicing/IsoformSwitchAnalyzeR/SCALPEL/SCALPEL/src/read_10X.R 1135_F_Multiome
  ln -s 1135_F_Multiome/outs/possorted_genome_bam.bam 1135_F_Multiome.bam
  ln -s 1135_F_Multiome/outs/possorted_genome_bam.bam.bai 1135_F_Multiome.bam.bai
  gunzip -c 1135_F_Multiome/outs/filtered_feature_bc_matrix/barcodes.tsv.gz > 1135_F_Multiome.barcodes

Command exit status:
  1

Command output:
  [1] "reading file..."

Command error:
  
  Attaching package: ‘dplyr’
  
  The following objects are masked from ‘package:data.table’:
  
      between, first, last
  
  The following objects are masked from ‘package:stats’:
  
      filter, lag
  
  The following objects are masked from ‘package:base’:
  
      intersect, setdiff, setequal, union
  
  Loading required package: SeuratObject
  Loading required package: sp
  
  Attaching package: ‘SeuratObject’
  
  The following objects are masked from ‘package:base’:
  
      intersect, t
  
  [1] "reading file..."
  10X data contains more than one type and is being returned as a list containing matrices of each type.
  Error in (function (..., row.names = NULL, check.rows = FALSE, check.names = TRUE,  : 
    arguments imply differing number of rows: 32285, 56742
  Calls: data.frame ... as.data.frame -> as.data.frame.list -> do.call -> <Anonymous>
  In addition: Warning messages:
  1: In asMethod(object) :
    sparse->dense coercion: allocating vector of size 3.2 GiB
  2: In asMethod(object) :
    sparse->dense coercion: allocating vector of size 5.6 GiB
  Execution halted

Work dir:
  /mnt/isilon/cristancho_data/Shikhar/Splicing/IsoformSwitchAnalyzeR/SCALPEL/SCALPEL/work/bc/030a4fc970a18736b9898b645e4485

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details

[mschilli87]

@guptas2005: Could you please share some information about your 1135_F_Multiome input? Some ls, head, tail, wc -l etc.
It is very difficult to help fixing a problem without being able to reproduce it ourselves.
Unfortunately, SCALPEL does not (yet) have good error handling, so its error messages alone are often not enough to understand what's going on. @Franzx7 is working on that but in the meantime we need better bug reports to be able to assist remotely.

[guptas2005]

@mschilli87 Of course, sorry about that! I have 16 input samples, 8 in each condition. Trying to eventually use this output in a program like IsoformSwtichAnalyzeR.

These are the folders inside of the cellranger output for the 1135_F_Multiome sample:

analysis
atac_cut_sites.bigwig
atac_fragments.tsv.gz
atac_fragments.tsv.gz.tbi
atac_peak_annotation.tsv
atac_peaks.bed
atac_possorted_bam.bam
atac_possorted_bam.bam.bai
cloupe.cloupe
filtered_feature_bc_matrix
filtered_feature_bc_matrix.h5
gex_molecule_info.h5
gex_possorted_bam.bam
gex_possorted_bam.bam.bai
per_barcode_metrics.csv
raw_feature_bc_matrix
raw_feature_bc_matrix.h5
summary.csv
web_summary.html

This is the contents of the filtered bc matrix folder:

ls -lh filtered_feature_bc_matrix

total 227M
-rwxrwx--- 1 guptas13 reslnusers  64K Apr 16  2021 barcodes.tsv.gz
-rwxrwx--- 1 guptas13 reslnusers 1.7M Apr 16  2021 features.tsv.gz
-rwxrwx--- 1 guptas13 reslnusers 226M Apr 16  2021 matrix.mtx.gz

Inside of matrix.mtx file:

zcat filtered_feature_bc_matrix/matrix.mtx.gz | head

%%MatrixMarket matrix coordinate integer general
%metadata_json: {"software_version": "cellranger-arc-1.0.1", "format_version": 2}
89027 13321 64351899
1 1 3
10 1 3
14 1 1
17 1 2
21 1 1
24 1 3
34 1 1

zcat filtered_feature_bc_matrix/matrix.mtx.gz | wc -l

64351902

Please do let me know what other information I can get to you, sorry about that!

[guptas2005]

I just ran :

zcat filtered_feature_bc_matrix/features.tsv.gz | cut -f3 | sort | uniq -c
  
32285 Gene Expression
56742 Peaks

It could not be working because it's not expecting multiomic data? Is there a workaround here?

[guptas2005]

@mschilli87 Sorry for these messy responses, but I believe I figured it out.

I manually extracted only the Gene Expression data from the cellranger matrix.mtx, features.tsv, and barcodes.tsv (this ignores the ATAC peaks data that also existed within the files). I then wrote them to new matrix.mtx, features.tsv, and barcodes.tsv files and directed SCALPEL to those.

Another note for anyone trying this on 10x Multiome data, the bam files produced are not in the expected "possorted_genome_bam.bam" and "possorted_genome_bam.bam.bai" naming convention. I had to manually rename them since they're default "gex_possorted_bam.bam" from 10x. I then copied these over into the new "cellranger output" folder that I indicated to SCALPEL in column 4 of the samplesheet.csv.

I was able to get a run to work, but I'm now running into a problem similar to [https://github.com//issues/20] where not every step of SCALPEL is running. I will report back with updates.

[mschilli87]

@guptas2005: Thank you for the feedback. That makes sense. I don't think SCALPEL has been tested on 10X multi-ome. I will leave this open so we either add a new format to handle this type of input or at least improve the documentation to point out this limitation and your work-around.
For your other issue, please comment on #20 if you are sure it's the same problem or open a new issue so we can track it separately. This issue here should focus on the Multiomics input issue only. Thank you.

[guptas2005]

Will do, thanks!