Gives an error in the parallel run

[kumarage]

_Hi,

I first used spatialGE with my windows RStudio which ran fine except the slow processing as it seems there is no support to run parallel in windows. Then I used CHTC/HTCondor based compute with "rocker/tidyverse:latest" apptainer. Everything ran fine till it got to STdiff function as shown below in the .error file._

"── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.5.1 ✔ tibble 3.2.1
✔ lubridate 1.9.4 ✔ tidyr 1.3.1
✔ purrr 1.0.4
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (http://conflicted.r-lib.org/) to force all conflicts to become errors
Registered S3 methods overwritten by 'registry':
method from
print.registry_field proxy
print.registry_entrRegistered S3 methods overwritten by 'registry':
method from
print.registry_field proxy
print.registry_entry proxy
y proxy
Registered S3 method overwritten by 'spaMM':
method from
[.predictions
Registered S3 method overwritten by 'spaMM':
method from
[.predictions
Warning message:
In parallel::mclapply(1:length(unique(combo_df[["samplename"]])), :
all scheduled cores encountered errors in user code"

And my .R file looks like below:

**library('spatialGE')
library('tidyverse')

data_files <- 'STdiff_DEG_files'
count_files <- list.files(data_files, full.names=T, pattern='counts')
coord_files <- list.files(data_files, full.names=T, pattern='mapping')
clin_file <- list.files(data_files, full.names=T, pattern='meta')

mbd1 <- STlist(rnacounts=count_files, spotcoords=coord_files, samples=clin_file)
mbd1 <- transform_data(mbd1)
mbd1 <- STclust(mbd1,
ks='dtc',
ws=0.025)

#Run spatial DEGs
deg <- STdiff(mbd1,
samples=c('slice1_WT','slice1_KO'),
w=0.025, k='dtc',
deepSplit=F,
topgenes=300,
sp_topgenes=0.2,
cores=24)**

[oospina]

Hello,

Thank you for reaching out and your interest in spatialGE.

At first look, one possibility is that the process crashes due to memory limits (may not be your case if your 24-core environment has also a lot of memory).

Also, can you please share the error output when running with cores=1 ?

Thank you!

[kumarage]

Hi,

Thanks for the quick response. It runs fine now. I think the issue was memory per core.
I am using STdiff function with Merfish data for ~4000 L5/6 neurons to compare WT and KO Spatially varying genes. It is very resource intensive.

Best!