Improve shared SQL database

[koppor]

• Manage connections: http://discourse.jabref.org/t/managing-shared-database-connections/289
• Command line access: http://discourse.jabref.org/t/command-line-access-to-sql-databases/288
• Make UI responsive for shared databases

Regarding the responsiveness:

1. Create an account and a "tiny turtle" database at https://www.elephantsql.com/
2. Connect to the database
3. Paste 100 entries
4. UI becomes unresponsive

[Luujauhe]

Me, @saschatran and @rohailg24 will be working on this issue as part of the
Software Engineering course at University of Basel, and we have a few questions.

Regarding "Manage connections":

1. Should JabRef save every remote database you connect to, or only the ones you specify that
   you want to save?
2. Should the user also be able to save a database without connecting to it?
3. Should the user be able to edit and/or remove already saved databases?
4. Should the password of saved databases also be saved or should it be asked upon
   switching shared databases? If yes, does JabRef have specific guidelines regarding encryption and security of sensitive data?
5. Is there any existing remote databases we can use to test the features once implemented or should
   we set up our own? In the second case, is there anything we should be aware of
   (e.g. required tables, specific database structure, ...) for JabRef to work properly?

Note: I use the term save a database as a shortcut for save the credentials of a database

[saschatran]

Hi I've got a question too.
Regarding "Make UI responsive for shared databases"

1. According to the instructions, the UI becomes unresponsive. Is the goal to make the UI responsive to a reasonable amount of entries?
2. If so, what would be a reasonable amount of entries?
3. Or is this issue about figuring out where the problem lies in? As in, there is already a function but it doesn't work as supposed. We'd be glad if you could specify this part of the issue.

Thanks in advance.

[rohailg24]

Hi I've also got a question regarding the Command line access issue.
We just want to make sure that we really understand the task.
Is our goal to extend the jabref -na function which can run queries against a remote database or is it not possible to do so?

Thanks in advance.

[saschatran]

Hi sorry to spam you full.

I've just tried the issue part about the unresponsive UI according to your description.
I've pasted like 500 entries into the Database and seems like the UI is still working.

Hence, I'm very unsure of what the issue is exactly about.
What exactly do you mean by unresponsive? Resp. what or when should I expect it?

Thanks for your time.

[calixtus]

@s-rousselle: Regarding questions 1-3
https://de.wikipedia.org/wiki/PuTTY

4. Should the password of saved databases also be saved or should it be asked upon
   switching shared databases? If yes, does JabRef have specific guidelines regarding encryption and security of sensitive data?

If the password is stored, jabref shouldn't ask for, except the password is wrong (provide a dialog with save password checkbox)
Until now, jabref stored the proxy password in plain text. Not good, but there wasn't much effort yet to improve that. If you have some time to implement it, go for it.

5. Is there any existing remote databases we can use to test the features once implemented or should
   we set up our own? In the second case, is there anything we should be aware of
   (e.g. required tables, specific database structure, ...) for JabRef to work properly?

Good questions. Wouldn't be to bad for testing issues, if we had a shared database we can test on. We'll talk about that in the next devcall. Meanwhile, set up your own remote database for testing.

@saschatran: You should be able to move the synchronizing stuff to a background task (jabref already offers infrastructure to follow these thanks to @btut). If there are parts of the task that would block the user from working on the main table, JabRef should display some kind of waiting animation. If there is a huge load of entries (10.000 or so, yes, we really have users with this number of entries). If there is an infinite loop it would probably be a bug, that needs fixing. Maybe @koppor can specify further.

@rohailg24 I believe the task would be to expand the existing cli of jabref to work with shared databases.

[btut]

You should be able to move the synchronizing stuff to a background task

I think the best thing to do here is to go for the same approach as for the pdf indexer. Have one background task with a queue of database queries. You can checkout the IndexingTaskManager to get an idea of how to do that.

[rohailg24]

Hi Thank you for your answer.
According to the instructions, we should automate the process of entries retrieval, is the task to import a Latex Document with all the entries in it or is a simple commandline output sufficient?
Regarding the export, what exactly does the user want to export? a Latex-Document? Or a simple input in the commandline

[calixtus]

I don't think that you understood the instructions right...

JabRef works on .bib files, not with latex documents. It already can sync via the gui entries with a remote database. Locally those entries are stored in a .bib file, remotely in a database. Syncing/importing/exporting already works in the gui and does not have to be improved (well, actually it could be improved, but thats not the concern of this issue).

This issue applies to improving

• the management of multiple databases (to switch between multiple database, to store credentials, to switch between different profiles etc.)
• making the already existing functionality of the gui accessible via the cli (syncing a .bib file with a remote database, running cleanup-formatters, etc.)
• improving the responsiveness of the gui while synchronizing a remote database with your local .bib file

[koppor]

The responsiveness should not be treated in the work of the course.

Thus, two issues remain:

• Manage connections: http://discourse.jabref.org/t/managing-shared-database-connections/289
• Command line access: http://discourse.jabref.org/t/command-line-access-to-sql-databases/288

1. Should JabRef save every remote database you connect to, or only the ones you specify that you want to save?

It should save everyone and offer deletion of existing ones.

2. Should the user also be able to save a database without connecting to it?

No, JabRef should do automatic saving

3. Should the user be able to edit and/or remove already saved databases?

Yes

4. Should the password of saved databases also be saved or should it be asked upon
   switching shared databases?

Yes

If yes, does JabRef have specific guidelines regarding encryption and security of sensitive data?

No. See #8055 (comment) for solution proposals. - Please try to use the proposed solution.

5. Is there any existing remote databases we can use to test the features once implemented or should
   we set up our own?

Please setup your own. See the first text of the issue: Use https://www.elephantsql.com/.

In the second case, is there anything we should be aware of (e.g. required tables, specific database structure, ...) for JabRef to work properly?

Did you try to connect to to https://www.elephantsql.com/? This should work instantly. Please try it and report back.

[koppor]

Regarding "Make UI responsive for shared databases"

This is very hard to work on and needs thorough testing. One should use the YourKit Java Profiler (https://www.yourkit.com/).

According to the instructions, the UI becomes unresponsive. Is the goal to make the UI responsive to a reasonable amount of entries?

Yes

If so, what would be a reasonable amount of entries?

10.000

Or is this issue about figuring out where the problem lies in?

Yez - we do not know about the root cause. We should observe that interaction with a shared database is slow.

[saschatran]

1. Is there any existing remote databases we can use to test the features once implemented or should
   we set up our own?

Please setup your own. See the first text of the issue: Use https://www.elephantsql.com/.

In the second case, is there anything we should be aware of (e.g. required tables, specific database structure, ...) for JabRef to work properly?

Did you try to connect to to https://www.elephantsql.com/? This should work instantly. Please try it and report back.

Yes, we did and it works. (We've already sent the question 1., right before we've figured out what the elephantsql does. Sorry for that.)

Thanks for the answers.

[Luujauhe]

It seems we are unable to run JabRef from the .bat file. When typing JabRef.bat (or without the .bat in Ubuntu) in a terminal, we get the following error:

Error occurred during initialization of boot layer
java.lang.module.FindException: Module org.jabref not found

It is possible that we missed a step, since we tried that directly in the build/scripts/ folder in IntelliJ. Could we get some help on how to run JabRef in CLI mode ?

[k3KAW8Pnf7mkmdSMPHz27]

since we tried that directly in the build/scripts/ folder in IntelliJ.

@s-rousselle generally speaking I'd suggest asking these kinds of questions in the Gitter, you are likelier to get a quick response there. I am not familiar enough with the build process of JabRef to give you instructions about what to change in the approach you are attempting, you can open the script itself and see what it tries to run. The instructions work for the downloadable app, but I don't think that is very helpful to you right now X)

I don't know the goals of your course, but if you want a quick fix that might allow you to proceed until someone gets back to you with a better method, I'd say use gradlew run with --args instead (e.g., ./gradlew run --args="-v"). Is this sufficient for now?

I think @btut did some CLI-related improvements, do you have a better workflow (IDE -> CLI)?

[btut]

Hi!
You mention you are using intellij. If you also use the Intellij build system you can create a run configuration to test the CLI.
I never used self generated binaries explicitly, but you could try if the pre-built binaries work for you. If they do, there may be a problem with your build.