JabRef 5 on Linux x86_64 Extreme Memory Pressure, System Hangs, Unusable

[PorcelainMouse]

JabRef version jabref-5.0.30388-1.x86_64 on

• I have tested the latest development version from http://builds.jabref.org/master/ and the problem persists

Steps to reproduce the behavior:

1. start JabRef
2. Watch Swap and Virt Mem
3. Move the mouse around in front JabRef window when it has focus

What happens is that JabRef Virtual Memory size is 111 GB! And just interacting with the GUI causes a lot of memory to page out, causing some programs to crash. I was running vmstat at the time and it crashed while 2 GB of swap got used in about 30 seconds. Mouse starts jumping and then sound will lock up. Then, the whole computer locked up for about another 30 seconds.

This has happened every time I use JabRef 5 beta packages.

System is Fedora 31, Xorg.

[AEgit]

JabRef 5.0-beta.403--2020-02-03--0d42a87
Windows 10 10.0 amd64
Java 13.0.2

Cannot confirm for current dev version of JabRef on Windows. So this problem might be Linux-specific (?).

[AEgit]

Maybe the following issues are connected:

#5882
#5912

[PorcelainMouse]

JabRef 5.0-beta.403--2020-02-03--0d42a87
Windows 10 10.0 amd64
Java 13.0.2

Cannot confirm for current dev version of JabRef on Windows. So this problem might be Linux-specific (?).

It is very likely Linux specific, but can you tell how big the VM is on Windows? That's my first question about that comparison.

In any case, you are probably using Oracle Java, and that is unlikely on Linux desktop systems. So, there are a lot of differences that need to be disentangled before any comparison is interpretable.

[PorcelainMouse]

I guess the difference between this bug and the other two, is just that I don't notice the problem right away. I don't notice much delay until the memory issue starts pushing huge numbers of pages out of main memory. And, there is no way to guarantee this will not happen unless the binary is altered so that less that 100 GB of virtual memory is needed. I have actually built java monoliths before, but I still don't understand all of it.

[AEgit]

Hmm, does that mean, that you do not always encounter this bug? I can test this also with a Linux installation, but if it is not easy to reproduce it might get difficult. Is triggering the issue dependant on your database/database size?

[Siedlerchr]

I updated some Java dependencies, could you please check the latest version from today? I'm just curious if it is better (

[wujastyk]

JabRef 5.0-beta.406--2020-02-05--6fda10a
Linux 5.3.0-28-generic amd64
Java 13.0.2

Still a problem.

I don't understand why the "version" above says Java 13.0.2. Is that jre packaged in the deb file? My system has OpenJDK 8 and 11:

% update-java-alternatives -l
java-1.11.0-openjdk-amd64      1111       /usr/lib/jvm/java-1.11.0-openjdk-amd64
java-1.8.0-openjdk-amd64       1081       /usr/lib/jvm/java-1.8.0-openjdk-amd64

Update: no, slowness of keystrokes is not such a problem as formerly. It's just when it starts up, I think. The CPU usage on four CPUs goes up briefly to 80% or 100% on startup:

then after a minute or two, things calm down a little and keystrokes are normal speed. It's a clear improvement.

[PorcelainMouse]

Hmm, I appreciate that CPU contention could cause slow downs, but that is not what I'm reporting here. At least I don't think it is, because I don't notice any CPU pressure. I'll try to get a graph, but for now, let's assume that is not this bug. My slowness is caused by swapping and memory exhaustion, but I have more than 2GB free when start jabref.

[PorcelainMouse]

I'm also seeing this:

java.lang.IllegalStateException: Not on FX application thread; currentThread = pool-337-thread-1
	at org.jabref.merged.module/com.sun.javafx.tk.Toolkit.checkFxUserThread(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.tk.quantum.QuantumToolkit.checkFxUserThread(Unknown Source)
	at org.jabref.merged.module/javafx.scene.Parent$3.onProposedChange(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.collections.VetoableListDecorator.setAll(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.collections.VetoableListDecorator.setAll(Unknown Source)
	at org.jabref.merged.module/javafx.scene.control.skin.LabeledSkinBase.updateChildren(Unknown Source)
	at org.jabref.merged.module/javafx.scene.control.skin.LabeledSkinBase.lambda$new$11(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.scene.control.LambdaMultiplePropertyChangeListenerHandler.lambda$new$1(Unknown Source)
	at org.jabref.merged.module/javafx.beans.value.WeakChangeListener.changed(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.binding.ExpressionHelper$SingleChange.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.set(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.set(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringProperty.setValue(Unknown Source)
	at org.jabref.merged.module/javafx.scene.control.Labeled.setText(Unknown Source)
	at org.jabref.merged.module/javafx.scene.control.skin.TabPaneSkin$TabHeaderSkin.lambda$new$2(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.scene.control.LambdaMultiplePropertyChangeListenerHandler.lambda$new$1(Unknown Source)
	at org.jabref.merged.module/javafx.beans.value.WeakChangeListener.changed(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.set(Unknown Source)
	at org.jabref.merged.module/javafx.beans.property.StringPropertyBase.set(Unknown Source)
	at org.jabref.merged.module/javafx.scene.control.Tab.setText(Unknown Source)
	at org.jabref/org.jabref.gui.JabRefFrame.updateAllTabTitles(Unknown Source)
	at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.doSave(Unknown Source)
	at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.save(Unknown Source)
	at org.jabref/org.jabref.gui.dialogs.AutosaveUIManager.listen(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.Subscriber$1.run(Unknown Source)
	at org.jabref.merged.module/com.google.common.util.concurrent.DirectExecutor.execute(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.Subscriber.dispatchEvent(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Unknown Source)
	at org.jabref.merged.module/com.google.common.eventbus.EventBus.post(Unknown Source)
	at org.jabref/org.jabref.logic.autosaveandbackup.AutosaveManager.lambda$listen$0(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

But, that is probably a different issue.

[wujastyk]

Before I start Jabref 5, I have about 3GB used out of 7GB; after starting Jabref it rises to about 4GB.

[PorcelainMouse]

But, I'm working more carefully, now, and I need to amend some statements.

1. It doesn't happen just moving the mouse around, that's just how I notice when it starts.
2. It seems to be just a large memory footprint and, when I use the last free page, a huge amount of pages get swapped out at once and that is what cases the hang. So, it is definitely related to my used memory.
3. but it seems to eat memory for no reason. If I leave it runnig, it ate another 1 GB while I wasn't even interacting with it.

I wonder if it's just too big. it uses about 3 times as much memory as my firefox, just when I start, now it's 8 times. All I did was create one new entry.

[PorcelainMouse]

Okay, more details. I tried to finish entering the rest of that one entry and JabRef ate all of my remaining swap before I could finish. I had to quit it, but it's hung and has a bunch of memory locked.

Whatever this is, it's a memory issue. I don't know why JabRef is using so much memory, but it is way to much. It's 1.3 GB in of RAM used right now and it supposed to be quitting.

I don't know how to help troubleshoot this further, but I'm happy to help. Just let me know what I can do.

[Siedlerchr]

JabRef ships with a packaged version of the JDK 13, a custom runtime image, which contains all necesary modules. If you want to run from code, you need to install a jdk 13.
@PorcelainMouse You could try to dig further in with the Java Profiler to locate the bottleneck,
https://www.yourkit.com/java/profiler/ (you can get a 15 days trial key)

[PorcelainMouse]

I updated some Java dependencies, could you please check the latest version from today? I'm just curious if it is better (

Yes, I will check ASAP.