[REVIEW]: treesiftr: An R package and server for viewing phylogenetic trees and data

[whedon]

Submitting author: @wrightaprilm (April Wright)
Repository: https://github.com/wrightaprilm/treesiftr
Version: v1.0.0
Editor: @juanklopper
Reviewer: @ethanwhite, @rachelss
Archive: 10.5281/zenodo.2541824

Status

Status badge code:

HTML: <a href="http://jose.theoj.org/papers/82b5b8c6223ec0735602bbf927045913"><img src="http://jose.theoj.org/papers/82b5b8c6223ec0735602bbf927045913/status.svg"></a>
Markdown: [![status](http://jose.theoj.org/papers/82b5b8c6223ec0735602bbf927045913/status.svg)](http://jose.theoj.org/papers/82b5b8c6223ec0735602bbf927045913)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@ethanwhite & @rachelss, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/jose-reviews/invitations

The reviewer guidelines are available here: https://jose.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @juanklopper know.

Review checklist for @ethanwhite

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v1.0.0)?
• Authorship: Has the submitting author (@wrightaprilm) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rachelss

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v1.0.0)?
• Authorship: Has the submitting author (@wrightaprilm) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @ethanwhite, it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[labarba]

@ethanwhite, @rachelss — Thank you for agreeing to review this submission for JOSE! We are happy to welcome you to our adventure in scholarly publishing of open-source educational software and learning resources.

JOSE has published seven papers so far 😃 — have a look: http://jose.theoj.org
Check our Reviewer Guidelines and let us know if you have any questions.

@juanklopper will be your handling editor: you're in good hands.

[juanklopper]

@ethanwhite, @rachelss a big thank you from me too. I look forward to your input with this submission. Let me know if there are any questions. If I can't help then @labarba will most definitely have the answers.

[ethanwhite]

@juanklopper - @wrightaprilm and I are co-authors on a Data Carpentry lesson (http://doi.org/10.5281/zenodo.570050). There are a large number of authors and I don't believe that this interferes with my objectivity, but it was in the last 4 years. I am reporting this to you as requested in the CoI link for your input.

[labarba]

Thanks for reporting this, @ethanwhite. I agree that this doesn't present conflict.

[ethanwhite]

OK, great. Thanks @labarba. I will proceed.

[ethanwhite]

A couple of questions/comments for @wrightaprilm from the "General checks" section:

License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)

Following the R packaging standard the LICENSE file does not include the actual license, only the year and copyright holder. The license is given in the DESCRIPTION file. One common approach to dealing with the conflicts between R's packaging standard and the more general open source standard of having the full license in the LICENSE file is to have the full license in LICENSE.md (which is included in .Rbuildignore) and keep LICENSE in the standard R format. This is rOpenSci's general strategy. I'm not sure what JOSS or JOSE's general approach is to this issue.

Version: Does the release version given match the repository release (v1.0.0)?

The repository is lacking a current release. If you're waiting to tag 1.0.0 until other comments have been satisfied that makes sense and I'll wait until it looks like things are wrapping up and then ping this issue again about tagging a release.

[wrightaprilm]

One common approach to dealing with the conflicts between R's packaging standard and the more general open source standard of having the full license in the LICENSE file is to have the full license in LICENSE.md (which is included in .Rbuildignore) and keep LICENSE in the standard R format. This is rOpenSci's general strategy. I'm not sure what JOSS or JOSE's general approach is to this issue.

This uncertainty is why the package is the way it is right now. I put the MIT license in the description, but wasn't sure where else that info needed to be, if anywhere.

The repository is lacking a current release. If you're waiting to tag 1.0.0 until other comments have been satisfied

Bingo!

[rachelss]

There are a couple of typos in paper.md - specifically [application]((https://wrightaprilm.shinyapps.io/treesiftr_app/) has an extra parenthesis in two places; the phrase "30-day paleobiological data " needs to be fixed, "non-paramtric" should be "non-parametric", "and use in " should be "which I use in". You might want to render with pandoc to check everything looks write prior to resubmission.

@juanklopper - I'd suggest that the submission requirements suggest submitters render paper.md with pandoc prior to submission in general as it's easier to proofread that way.

To install treesiftR I need ggtree. To install ggtree I need BiocManager. To install BiocManager I need R >= v3.5 . Alternatively I was able to install ggtree on my older version of R with older version of bioconductor with BiocInstaller::biocLite("ggtree"). I should probably keep everything up to date, and I realize listing dependencies is sufficient, but I bet I'm not the only one who doesn't update so including these extra instructions would help out.

[labarba]

@rachelss — Our dear bot whedon generates the PDF for us right here in this issue. Scroll up and you will see:

👉 Check article proof 📄 👈

[rachelss]

@wrightaprilm I could use a bit more explanation in the Instructor's Guide and the Shiny app. Or maybe I missed a piece of documentation? I went straight to Guide and app.

After navigating to the app I am instructed to subset a phylogenetic matrix. Where did the matrix come from? Maybe you could explain the data a bit first. That would help understand when I'm setting start = 1 what that means. You might put similar text on the app itself for users.

The character matrix on the righthand side of the app is not intuitive. Some words in the app to explain what these blocks are would help. Also, just having the blocks on the right doesn't make it obvious that the root / internal nodes are one color or the other - can you put colors of ancestral states on the tree?

Is the ability to view the likelihood score useful? I can see the tree is less likely with more characters but really you want to compare likelihoods for a single dataset not across datasets, and students can't do that here. But maybe parsimony isn't sufficient to really explain phylogeny estimation - it would depend on the class.

How much can Shinyapps.io handle? How many students could be using the app simultaneously? How many students across the country could be using it in a class (ie if a bunch of classes start using this will the app no longer be available)? Just wondering if there will end up being limitations that will make this difficult to use widely. You might need to provide guidance for users to prevent the app from crashing.

@juanklopper - is this how we're supposed to write these reviews?

[wrightaprilm]

Thanks, @rachelss, for these comments. I'll start working on them.

With re:

I'd suggest that the submission requirements suggest submitters render paper.md with pandoc prior to submission in general as it's easier to proofread that way.

I agree that this would be a useful addition to the author guidelines. I rendered in RStudio, which has its own idiosyncrasies. With a research paper submission, we normally think about grabbing the Latex template and rendering as you go, so that is a suggestion that would feel natural to people. Including a link to the template or preferred pandoc settings is something folks should have no real trouble with.

[juanklopper]

@wrightaprilm I'm back from all my duties as external examiner and wanted to know how you are getting along. Big thanks to @rachelss Excellent insight and suggestions.

[wrightaprilm]

I think we're good, thanks! Just waiting on @ethanwhite, in case there are conflicts between the two reviews.

Which is fine, I think we've all had some end-of-semester stuff going on that would make revising hard ;)

[ethanwhite]

Here's my review. Thanks for your patience and understanding @wrightaprilm. The end of the semester is indeed an... interesting time.

Overall, great work and really useful. I think this will be a great addition to a number of workshops and classrooms. For every checkbox that I haven't checked I've provided a description of what I think can be improved to make this ready to go.

General Checks

• License: The license file follows R's guidelines, but as a result the repository does not include an actual license.
• Version: The versions do not currently match because v1.0.0 does not currently exists. The author plans to release this version once the rest of the review is complete.

Documentation

• Usage: There are examples of how to use the module and questions that can be asked of students along with the answers. These are both very helpful. I think it would be useful in both the README and the Instructions guide to include some documentation designed to explain specifically how someone would adopt the module in a teaching setting. This is covered a bit in the JOSE paper and you get the idea that the learning material is lab exercise focused from the vignettes, but I'd suggest also putting some of those ideas front and center in the README and Instructors Guide so that folks more immediately understand the goals of the package and learing materials.
• I'd also suggest adding a pdf of the Instructors Guide and linking to it directly from the README so that it is easy to find and read for someone first encountering the package online.
• Community guildelines: There is a code of conduct (which is awesome!) but I didn't find information on how to contribute or seek support.

Pedagogy / Instructional design

• Learning Objectives: are a little difficult to determine from the current documentation. I understand that they are about helping students understand tree construction and phylogeny more broadly, but it would be helpful to lay this out a little more explicitly. I think a second short paragraph in the Introduction of the Instructors guide should be sufficient.

JOSE paper

• Use in classroom or other settings: I think just a couple of additional sentences describing how to adopt this material would make a big differences for helping make clear to readers exactly how they could use the material.
• Could someone else teach with this module: I think the module needs just a little more documentation. Once the changes described above are made I think this criteria will be satisfied.
• References: Lewis 2001 is missing the DOI: 10.1080/106351501753462876

[wrightaprilm]

Great, thanks both. Doesn't look like there's too much conflict between these two reviews. So revisions should go like dominoes, eh?

@juanklopper - what do I have for timing here? Am I OK to come back with these notes addressed in January?

[labarba]

👋 @wrightaprilm Happy New Year! You can proceed with the revision at your own pace, but it would be great to keep it going. Keep us posted!

[wrightaprilm]

Happy New Year, y'all.

I've responded to most of the points raised by reviewers. I'm still working on @rachelss's excellent suggestions about ancestral states + random trees, and should finish it tomorrow. I made the rest of the suggested edits, but made notes where I found suggestions confusing and/or wasn't sure how to resolve them within journal style.

Edit: Thanks much to all four of you; this has been really helpful!

Rachel's review:

I could use a bit more explanation in the Instructor's Guide and the Shiny app. Or maybe I missed a piece of documentation? I went straight to Guide and app.

After navigating to the app I am instructed to subset a phylogenetic matrix. Where did the matrix come from? Maybe you could explain the data a bit first. That would help understand when I'm setting start = 1 what that means. You might put similar text on the app itself for users.

I added some text to the instructor's guide, and added popify objects to the web interface to provide more information about the options, and a citation for the dataset.

The character matrix on the righthand side of the app is not intuitive. Some words in the app to explain what these blocks are would help. Also, just having the blocks on the right doesn't make it obvious that the root / internal nodes are one color or the other - can you put colors of ancestral states on the tree?

This is a fantastic idea. I'm still working on this, but the short answer should be yes.

Is the ability to view the likelihood score useful? I can see the tree is less likely with more characters but really you want to compare likelihoods for a single dataset not across datasets, and students can't do that here. But maybe parsimony isn't sufficient to really explain phylogeny estimation - it would depend on the class.

Ideally, you're correct. But I'm likely to add to this package later. We have a paper in the works on some alternative models of morphological evolution, and something that I'm likely to expand into in the future is adding more buttons to see how the assumptions made about the data change the likelihood score we compute for the data. The differences are large enough that they can be seen on a single character.

How much can Shinyapps.io handle? How many students could be using the app simultaneously? How many students across the country could be using it in a class (ie if a bunch of classes start using this will the app no longer be available)? Just wondering if there will end up being limitations that will make this difficult to use widely. You might need to provide guidance for users to prevent the app from crashing.

Thank you, added this info!

Ethan's:

License: The license file follows R's guidelines, but as a result the repository does not include an actual license.

Could I have some editor guidance on this one? I followed R conventions (LICENSE file + link in the DESCRIPTION to a license), but if there's something else required, I'm happy to provide it.

I'd also suggest adding a pdf of the Instructors Guide and linking to it directly from the README so that it is easy to find and read for someone first encountering the package online.

Good plan! Done.

Community guildelines: There is a code of conduct (which is awesome!) but I didn't find information on how to contribute or seek support.

Good catch - I put these in the .github folder, which I then forgot to commit and push. I moved the CONDUCT file in here, too, so it pops up when someone opens an issue or PR.

Question on this - rOpenSci allows you to put an email for the organization, if people need to report that a maintainer (i.e. me) is the problem on a code of conduct issue. Does JOSE have a similar email address? I didn't see one, but it's possible I did not look in the right place.

Learning Objectives: are a little difficult to determine from the current documentation. I understand that they are about helping students understand tree construction and phylogeny more broadly, but it would be helpful to lay this out a little more explicitly. I think a second short paragraph in the Introduction of the Instructors guide should be sufficient.

Good idea. I added a list of learning objectives to each subsection of the instructor guide, and which questions addressed each.

Usage: I think it would be useful in both the README and the Instructions guide to include some documentation designed to explain specifically how someone would adopt the module in a teaching setting. This is covered a bit in the JOSE paper and you get the idea that the learning material is lab exercise focused from the vignettes, but I'd suggest also putting some of those ideas front and center in the README and Instructors Guide so that folks more immediately understand the goals of the package and learing materials.

&

Use in classroom or other settings: I think just a couple of additional sentences describing how to adopt this material would make a big differences for helping make clear to readers exactly how they could use the material.

I'm not quite sure how to get at this differently than I have. I added one sentence on this at the end of the second paragraph of the statement of need, and one to the end of the introduction to the instructor's guide.

References: Lewis 2001 is missing the DOI: 10.1080/106351501753462876

Good catch.

[wrightaprilm]

All right, I implemented Rachel's idea to generate random trees to show how traits score differently when phylogenetic relationships are not informative with respect to trait evolution.

With respect to the question of ancestral states: ggtree is developed quite separately from the rest of the phylogenetics R space. I think the appropriate way to handle this feature is to make a pull request against Phangorn to add an additional output type for their ancestral state estimation machinery. Would it be all right if I put this suggestion on a longer-term feature request timetable, as opposed to immediately for review? I'm happy to do it, but with involving additional developers on other projects, there is a lot out of my control on this.

Again, thanks all four of you.

[ethanwhite]

LGTM! Pending editorial input on the license (my personal take on the best compromise solution is above in the discussion related to adding a LICENSE.md file) and the version being updated to 1.0.0 I think this is ready to go. Nice work @wrightaprilm!

[labarba]

👋 @rachelss — It sounds like the author has addressed a bunch of review comments. Can you go over things and let us know here if your comments have been addressed and you are ready to recommend publication? Thanks!

[labarba]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[labarba]

@wrightaprilm You have an extra paren that breaks the link in the beginning of the second paragraph.

[wrightaprilm]

Let's try it!

[wrightaprilm]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[wrightaprilm]

OK, had to fix one citation to R core, but looks fine. Fingers crossed!

[labarba]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof 👉 openjournals/jose-papers#24

If the paper PDF and Crossref deposit XML look good in openjournals/jose-papers#24, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[wrightaprilm]

OK, looks good!

[labarba]

Nope ... some DOI links are broken. See screenshot.

[arfon]

I think bibtex expects doi fields in paper.bib to be of the form

10.1662/0002-7685(2007)69[71:csmaet]2.0.co;2 

not 

https://doi.org/10.1662/0002-7685(2007)69[71:csmaet]2.0.co;2

Entries like this one need updating:
https://github.com/wrightaprilm/treesiftr/blob/master/paper.bib#L181

[wrightaprilm]

Right, just went ahead and stripped off the `https' wrappers. Will re-render.

[wrightaprilm]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[labarba]

Checking all the links and ... http://ggplot2.org shows "not found"?

[labarba]

Did you want this URL instead?
https://ggplot2.tidyverse.org

[wrightaprilm]

I changed the link in my .bib. That's from the maintainer's own bibtex entry on their site. I should probably make a pull request and let them know its broken. Good catch!

[labarba]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof 👉 openjournals/jose-papers#25

If the paper PDF and Crossref deposit XML look good in openjournals/jose-papers#25, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[labarba]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSE! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.jose.00035 jose-papers#26
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/jose.00035
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

[labarba]

Congratuations, @wrightaprilm, your paper is published! 🎉
(you made us work hard for this one, phew!)

Big thank you to your handling editor, @juanklopper
and your reviewers, @ethanwhite and @rachelss
🙏

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://jose.theoj.org/papers/10.21105/jose.00035/status.svg)](https://doi.org/10.21105/jose.00035)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/jose.00035">
  <img src="https://jose.theoj.org/papers/10.21105/jose.00035/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://jose.theoj.org/papers/10.21105/jose.00035/status.svg
   :target: https://doi.org/10.21105/jose.00035

This is how it will look in your documentation:

We need your help!

Journal of Open Source Education is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://jose.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3ODA=

[wrightaprilm]

Thanks so much @juanklopper, @rachelss, @ethanwhite and @labarba!

[labarba]

https://twitter.com/JOSE_TheOJ/status/1086026521846996994

[juanklopper]

Well done @wrightaprilm and thank you to everyone for their input.