[REVIEW]: UralicNLP: An NLP library for Uralic Languages

[whedon]

Submitting author: @mikahama (Mika K Hämäläinen)
Repository: https://github.com/mikahama/uralicNLP
Version: 1.1.1-joss
Editor: @gkthiruvathukal
Reviewer: @gweissman
Archive: 10.5281/zenodo.2668061

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/4a2fe778b78f1845bea0f18367f8e8ab"><img src="http://joss.theoj.org/papers/4a2fe778b78f1845bea0f18367f8e8ab/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/4a2fe778b78f1845bea0f18367f8e8ab/status.svg)](http://joss.theoj.org/papers/4a2fe778b78f1845bea0f18367f8e8ab)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@gweissman, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @gkthiruvathukal know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @gweissman

Conflict of interest

• As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Version: 1.1.1-joss
• Authorship: Has the submitting author (@mikahama) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @gweissman it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[arfon]

👋 @gweissman - how are you getting on with your review?

[gweissman]

@arfon This my first time reviewing for JOSS so still trying to learn the checklist, which I have completed above. Additional comments I would like to include in the review are:

• Most of the code examples in the README.md file work with the following exception: when I type cg.disambiguate(tokens) I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/uralicNLP/cg3.py", line 39, in disambiguate
    cg_conv = Popen(["cg-conv" ,"-f"], stdout=PIPE, stdin=p1.stdout)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'cg-conv': 'cg-conv'

• When I type uralicApi.download("fin") the download appears to work correctly, and a status message is completed at the end of each download. These took a while and I would suggest providing a status update prior to initiating the download rather than after it has completed so the user knows the program is not frozen.

[mikahama]

@gweissman It seems that you do not have VISL CG-3 installed. It is required for constraint grammars and unfortunately cannot be installed over pip as a dependency. This requirement has been stated in the readme file.

I think that adding an indicator (or a progress bar) for downloading the models is a great idea. Thanks for the suggestion.

[gweissman]

@mikahama You are correct this is mentioned in the README.md file. I have nothing further to add. Congratulations on what appears to be a very useful software package.

[gkthiruvathukal]

@gweissman Just checking in on whether your review is complete or not. I still see a few items unchecked on the checklist.

[gweissman]

@gkthiruvathukal I have completed my review. Some of the check boxes I am unable to adequately assess. For example, I don't speak or read any of the languages in question so I can't make claims about functionality of the software.

• Not all of the references contained a DOI
• I did not see explicit community guidelines for participation
• I am unable to determine if there are automated tests in this software package

[gkthiruvathukal]

@mikahama Can you please follow up with how you intend to address the 3 issues reported by @gweissman? All three of these issues should be addressed before we can proceed further.

[mikahama]

As far as DOIs go, I have improved the situation with the latest version of UralicNLP. It is now possible to use uralicApi.model_info("fin") which will output information about the authors, copyright and such details. At least this will output a url to the Giellatekno repository where the FST and CG models are from. I have also gathered as much information as possible from the files in the Giellatekno repository in the model_info function. Sadly, it is quite common that the AUTHORS and LICENCE files are empty (see https://victorio.uit.no/langtech/trunk/langs/nob/ for example)

I have added CONTRIBUTING.md to the repository.

Also, I have improved the coverage of the automated tests (see https://github.com/mikahama/uralicNLP/blob/master/travis_test.py), however I do admit that more work should be done here to cover all the functionalities.

[gweissman]

@gkthiruvathukal I have no further comments. @mikahama has responded adequately to the points raised in my review.

[gkthiruvathukal]

Thanks, @gweissman. And thanks, @mikahama for addressing all review feedback.

I have gone through the checklist, and I'm ready to accept.

@mikahama Can you please archive the accepted version of the package and provide an associated DOI (e.g., via Zenodo)?

[mikahama]

It is now on Zenodo https://zenodo.org/record/2668061

[gkthiruvathukal]

@whedon set 10.5281/zenodo.2668061 as archive

[whedon]

OK. 10.5281/zenodo.2668061 is the archive.

[gkthiruvathukal]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[gkthiruvathukal]

@openjournals/joss-eics I'm ready to move toward acceptance of this submission. The archive has been set. I've checked the article proofs and everything appears to be fine.

[labarba]

hi @mikahama — the JOSS paper format has the software repository on the left-margin of the first page, therefore we ask that you don't add the repository again in the text/footnote. Can you fix?

[labarba]

Also, can you edit the Zenodo metadata so the title matches the JOSS paper? You could also add your ORCID.

[labarba]

@whedon set 1.1.1-joss as version

[whedon]

OK. 1.1.1-joss is the version.

[mikahama]

@labarba Hi, I removed the Github link in the footnote and edited the Zenodo metadata.

[labarba]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1007/978-3-642-40486-3_4 may be missing for title: HFST a system for creating NLP tools

INVALID DOIs

- None

[whedon]

Check final proof 👉 openjournals/joss-papers#676

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#676, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[labarba]

@mikahama — I'm going to ask for a little clean-up on your bibliography. When you have acronyms, like NLP, you should use caps protection only around that string, i.e., {NLP}, and not the whole title. For example, this line will override the bibliography style for title capitalization by caps protection on the whole title:
https://github.com/mikahama/uralicNLP/blob/677a54022a21abb1d863de0b78f7bc8514b95232/paper/paper.bib#L3

For two entries, I found you can add a DOI or permalink:

• yokohama add permalink http://hdl.handle.net/10138/282733
• linden2013hfst add DOI https://doi.org/10.1007/978-3-642-40486-3_4

[mikahama]

@labarba I fixed the bibliography

[labarba]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.01345 joss-papers#677
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01345
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

[labarba]

Congratulations, @mikahama, your JOSS paper is published! 🎉

Huge thanks to our editor: @gkthiruvathukal, and reviewer: @gweissman — we could not do this without you 🙏

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01345/status.svg)](https://doi.org/10.21105/joss.01345)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01345">
  <img src="http://joss.theoj.org/papers/10.21105/joss.01345/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01345/status.svg
   :target: https://doi.org/10.21105/joss.01345

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss