ERROR: Ran out of memory during hit loading; try re-running with increased -Xmx option.

[willluo123]

Hi, when I perform SEM with my mnase-seq data, i meet this error, the code is as follows:

Processing HitLoaders for: experiment rep1
Loading data from experiment:rep1:signaljava.lang.OutOfMemoryError: Java heap space
at java.base/java.util.regex.Pattern.compile(Pattern.java:1068)
at java.base/java.lang.String.replaceFirst(String.java:2081)
at org.seqcode.deepseq.hitloaders.SAMFileHitLoader.sourceAllHits(SAMFileHitLoader.java:71)
at org.seqcode.deepseq.experiments.HitCache.initialize(HitCache.java:227)
at org.seqcode.deepseq.experiments.HitCache.(HitCache.java:186)
at org.seqcode.deepseq.experiments.Sample.initializeCache(Sample.java:85)
at org.seqcode.deepseq.experiments.ExperimentManager.(ExperimentManager.java:125)
at org.seqcode.deepseq.experiments.ExperimentManager.(ExperimentManager.java:52)
at org.seqcode.projects.sem.SEM.main(SEM.java:179)
Ran out of memory during hit loading; try re-running with increased -Xmx option.

[yztxwd]

Hi, -Xmx is the option comes with java, if you install sem through conda, you can directly pass it to sem command, it should be able to parse java memory related options, example:

sem -Xmx100G ...

if you use the jar file directly, you can use

java -Xmx100G -jar sem.jar ...

Let me know if there is any further issue!

[willluo123]

Thank you, this change goes well. But I meet another problem, as followed:
ERROR: Exception when analyzing regionchr10:3498744-3548943
ERROR: Exception when analyzing regionchr10:3548944-3599043
ERROR: Exception when analyzing regionchr10:3449444-3498743
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.seqcode.projects.sem.mixturemodel.BindingEM.EM_MAP(BindingEM.java:551)
at org.seqcode.projects.sem.mixturemodel.BindingEM.train(BindingEM.java:329)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.analyzeWindowEM(BindingMixture.java:441)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.run(BindingMixture.java:356)
at java.base/java.lang.Thread.run(Thread.java:829)
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.seqcode.projects.sem.mixturemodel.BindingEM.EM_MAP(BindingEM.java:551)
at org.seqcode.projects.sem.mixturemodel.BindingEM.train(BindingEM.java:329)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.analyzeWindowEM(BindingMixture.java:441)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.run(BindingMixture.java:356)
at java.base/java.lang.Thread.run(Thread.java:829)
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.seqcode.projects.sem.mixturemodel.BindingEM.EM_MAP(BindingEM.java:551)
at org.seqcode.projects.sem.mixturemodel.BindingEM.train(BindingEM.java:329)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.analyzeWindowEM(BindingMixture.java:441)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.run(BindingMixture.java:356)
at java.base/java.lang.Thread.run(Thread.java:829)
ERROR: Exception when analyzing regionchr10:3399044-3449443
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.seqcode.projects.sem.mixturemodel.BindingEM.EM_MAP(BindingEM.java:551)
at org.seqcode.projects.sem.mixturemodel.BindingEM.train(BindingEM.java:329)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.analyzeWindowEM(BindingMixture.java:441)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.run(BindingMixture.java:356)
at java.base/java.lang.Thread.run(Thread.java:829)
ERROR: Exception when analyzing regionchr10:3199344-3248843
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.seqcode.projects.sem.mixturemodel.BindingEM.EM_MAP(BindingEM.java:551)
at org.seqcode.projects.sem.mixturemodel.BindingEM.train(BindingEM.java:329)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.analyzeWindowEM(BindingMixture.java:441)
at org.seqcode.projects.sem.mixturemodel.BindingMixture$BindingMixtureThread.run(BindingMixture.java:356)
at java.base/java.lang.Thread.run(Thread.java:829)

[yztxwd]

Thanks for reporting the error, I can't exactly tell the cause of the error from traceback. could you provide the command you used and the full log information? Also if it's possible sharing the input files would be really helpful too.