Importer: JabRef stops responding (hangs) during PDF Import

[koobs]

JabRef Version

JabRef 5.2--2020-12-14--e9be339
Windows 10 10.0 amd64 
Java 14.0.2

• Mandatory: I have tested the latest development version from http://builds.jabref.org/master/ and the problem persists

Summary

JabRef stops responding ("hangs") while processing files during import and provides no user-visible mechanism for progress. The length of the hang is proportional to the number of files selected, particularly evident during the creation of a new library, where many files are more likely to be imported.

The issue is also significantly exacerbated given imported file contents and other metadata are processed (for example: DOI and other lookups in PDF's) using remote service calls, which can and do present and add significant latencies to the processing time.

The issue is more likely to impact two user segments: new/newer users and users converting from other reference managers, where existing file collection sizes may be substantial, and as such presents a significant user experience issue.

A PDF collection of 264 files was used during my testing.

Steps to Reproduce

1. Lookup -> Search for unlinked files
2. Select N > 1 files for import -> Import
3. Observe total processing time
4. Repeat steps 2-3 for higher file counts (N)
5. Observe that the Import dialog and JabRef main window cannot be interacted with (see screenshots below) until Import process completes.

Additional Information (Screenshot)

Proposed Solution

• If Import processing is not currently executed off main thread, execute it off main thread
• On Import execution (start), add a visual progress mechanism to the Import dialog (see screenshot example below)

(Optional) Allow in-progress import to be cancelled (needs UI/UX)

• Example: Change Import button control to Cancel once Import is started

(Optional) Use the Import dialog tree control elements (of selected files) to indicate progress

Example:

• For each folder item, change ☑ <Folder Name> (N files) to ☐ <Folder Name> (Processing X of N files). Change ☐ to ☑ once each folder processing is complete.
• For each child item (file), change ☑ <filename> to ☐ <filename> (Processing...). Change ☐ to ☑ once each file processing is complete

[Siedlerchr]

Thanks for the report! This is indeed a problem, I looked into the code. The import is executed on the main javafx thread:

jabref/src/main/java/org/jabref/gui/externalfiles/FindUnlinkedFilesDialog.java

Lines 192 to 195 in c0153bd

	setResultConverter(buttonPressed -> {
	if (buttonPressed == buttonTypeImport) {
	startImport();
	} else {

Needs to be a proper background task.

[koobs]

@Siedlerchr My pleasure, you're welcome, and thank you for the assessment :)

[Siedlerchr]

We completely refactored the dialog now. You can test it in the latest development version (currently building)

We would like to ask you to use a development build from https://builds.jabref.org/master and report back if it works for you. Please remember to make a backup of your library before trying-out this version.

[koobs]

@Siedlerchr Thank you so much. Will test latest build and report back on all related (and closed) issues