Author name with special characters breaks JabRef

[carlinmack]

JabRef version on JabRef 5.0-beta.334--2020-01-08--cafdb01 - Windows 10 10.0 amd64 - Java 13.0.1

• I have tested the latest development version from http://builds.jabref.org/master/ and the problem persists

Steps to reproduce the behavior:

1. Add the following doi 10.1016/j.cosrev.2017.10.002
2. Use the search button to download the file
3. It downloads M�ntyl�2018 - The Evolution of Sentiment Analysis}A Review of Research Topics, Venues, and Top Cited Papers.pdf - note the special characters.
4. When you try to save jabref logs an error

Log File

org.jabref.logic.exporter.SaveException: Problems saving:
  at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.saveDatabase(Unknown Source)
  at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.doSave(Unknown Source)
  at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.save(Unknown Source)
  at org.jabref/org.jabref.gui.exporter.SaveDatabaseAction.save(Unknown Source)
  at org.jabref/org.jabref.gui.BasePanel.runCommand(Unknown Source)
  at org.jabref/org.jabref.gui.actions.OldDatabaseCommandWrapper.execute(Unknown Source)
  at org.jabref/org.jabref.gui.actions.JabRefAction.lambda$new$2(Unknown Source)
  at org.jabref.merged.module/org.controlsfx.control.action.Action.handle(Unknown Source)
  at org.jabref.merged.module/org.controlsfx.control.action.Action.handle(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEventImpl(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEvent(Unknown Source)
  at org.jabref.merged.module/javafx.event.Event.fireEvent(Unknown Source)
  at org.jabref.merged.module/javafx.scene.Node.fireEvent(Unknown Source)
  at org.jabref.merged.module/javafx.scene.control.Button.fire(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.scene.control.behavior.ButtonBehavior.mouseReleased(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.scene.control.inputmap.InputMap.handle(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.CompositeEventHandler$NormalEventHandlerRecord.handleBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEventImpl(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.event.EventUtil.fireEvent(Unknown Source)
  at org.jabref.merged.module/javafx.event.Event.fireEvent(Unknown Source)
  at org.jabref.merged.module/javafx.scene.Scene$MouseHandler.process(Unknown Source)
  at org.jabref.merged.module/javafx.scene.Scene.processMouseEvent(Unknown Source)
  at org.jabref.merged.module/javafx.scene.Scene$ScenePeerListener.mouseEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(Unknown Source)
  at java.base/java.security.AccessController.doPrivileged(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$2(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(Unknown Source)
  at org.jabref.merged.module/com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.glass.ui.View.handleMouseEvent(Unknown Source)
  at org.jabref.merged.module/com.sun.glass.ui.View.notifyMouse(Unknown Source)
  at org.jabref.merged.module/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
  at org.jabref.merged.module/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
  at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Error in field 'FILE': Unescaped '}' character without opening bracket ends string prematurely. Field value: :Mntyl 2018 - The Evolution of Sentiment Analysis}A Review of Research Topics, Venues, and Top Cited Papers.PDF:PDF;:M�ntyl�2018 - The Evolution of Sentiment Analysis}A Review of Research Topics, Venues, and Top Cited Papers.PDF:PDF
  at org.jabref/org.jabref.logic.bibtex.BibEntryWriter.writeField(Unknown Source)
  at org.jabref/org.jabref.logic.bibtex.BibEntryWriter.writeRequiredFieldsFirstRemainingFieldsSecond(Unknown Source)
  at org.jabref/org.jabref.logic.bibtex.BibEntryWriter.write(Unknown Source)
  at org.jabref/org.jabref.logic.exporter.BibtexDatabaseWriter.writeEntry(Unknown Source)
  at org.jabref/org.jabref.logic.exporter.BibDatabaseWriter.savePartOfDatabase(Unknown Source)
  at org.jabref/org.jabref.logic.exporter.BibDatabaseWriter.saveDatabase(Unknown Source)
  ... 62 more
Caused by: org.jabref.logic.bibtex.InvalidFieldValueException: Unescaped '}' character without opening bracket ends string prematurely. Field value: :Mntyl 2018 - The Evolution of Sentiment Analysis}A Review of Research Topics, Venues, and Top Cited Papers.PDF:PDF;:M�ntyl�2018 - The Evolution of Sentiment Analysis}A Review of Research Topics, Venues, and Top Cited Papers.PDF:PDF
  at org.jabref/org.jabref.logic.bibtex.LatexFieldFormatter.checkBraces(Unknown Source)
  at org.jabref/org.jabref.logic.bibtex.LatexFieldFormatter.formatAndResolveStrings(Unknown Source)
  at org.jabref/org.jabref.logic.bibtex.LatexFieldFormatter.format(Unknown Source)
  ... 68 more

5. This would be okay as you could delete the lines from Jabref, but it leaves the bottom of the file looking like this

...

@Article{M�ntyl�2018,
  author    = {Mika V. M�ntyl� and Daniel Graziotin and Miikka Kuutila},
  journal   = {Computer Science Review},
  title     = {The evolution of sentiment analysis{\textemdash}A review of research topics, venues, and top cited papers},
  year      = {2018},
  month     = {feb},
  pages     = {16--32},
  volume    = {27},
  doi       = {10.1016/j.cosrev.2017.10.002},
  file      = 

5. removing the categories and breaking the jabref file
6. On opening the jabref file you get the following error
   

[AEgit]

JabRef 5.0-beta.388--2020-01-29--eb988b6
Windows 10 10.0 amd64
Java 13.0.2

Can confirm this issue.

1. Use Ctrl + N to generate a new entry
2. Select ID type "DOI" and copy "10.1016/j.cosrev.2017.10.002" into the ID field
3. Click "Generate"
4. The generated item will contain question marks instead of umlaut "ä" in the bibtex key and the author's name

Note, that this issue does not appear, if you use the "Web search" feature, with Google Scholar selected. If you enter the doi "10.1016/j.cosrev.2017.10.002" there, click "Search" and then "Import entries", the entry will be imported without question marks and with the umlaut properly formatted following bibtex standard.

[Siedlerchr]

I checked the http request to doi.org output, that is correct.
so somewhere in JabRef the wrong encoding is taken

@article{M_ntyl__2018,
	doi = {10.1016/j.cosrev.2017.10.002},
	url = {https://doi.org/10.1016%2Fj.cosrev.2017.10.002},
	year = 2018,
	month = {feb},
	publisher = {Elsevier {BV}},
	volume = {27},
	pages = {16--32},
	author = {Mika V. Mäntylä and Daniel Graziotin and Miikka Kuutila},
	title = {The evolution of sentiment analysis{\textemdash}A review of research topics, venues, and top cited papers},
	journal = {Computer Science Review}
}

[bernhard-kleine]

Using JabFox in win10/firefox the file could be imported without problems. In the process of doi-import there is obviously a problem with coding At least the problematic name could be copy-pasted without any problem. On my win10 machine everything is utf8.

[Siedlerchr]

I can no longer reproduce the problem, I updated the java version to 13.0.2 and from the changelog it seems they fixed some issues regarding network/http requests. So it could have been related to that.
I also checked the DOI from #5900. Works fine.

Could you please try with the latest master and report back if it works?
Java Version in the about dialog should be 13.0.2
Could you please try the latest snapshot?
http://builds.jabref.org/master/

[AEgit]

JabRef 5.0-beta.409--2020-02-09--6a9c915
Windows 10 10.0 amd64
Java 13.0.2

Cannot confirm, the issue persists. Have you tried the following DOI (as suggested by the thread starter):

10.1016/j.cosrev.2017.10.002

?

[Siedlerchr]

I could partially reproduce it now with the installer version. Running from code seems to work. There must be something wrong with the encoding in the preferences. Weird.