[REVIEW]: The Gene School: Metagenomics

[whedon]

Submitting author: @koadman (Aaron Darling)
Repository: https://github.com/thegeneschool/metagenomics
Branch with paper.md (empty if default branch):
Version: v0.0.1
Editor: @emckiernan
Reviewers: @pschloss, @pvanheus, @fbidu
Archive: Pending

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/ad5d599cf8274c9ab2dd3d6f0622a639"><img src="https://jose.theoj.org/papers/ad5d599cf8274c9ab2dd3d6f0622a639/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/ad5d599cf8274c9ab2dd3d6f0622a639/status.svg)](https://jose.theoj.org/papers/ad5d599cf8274c9ab2dd3d6f0622a639)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@hughshanahan & @pschloss, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @emckiernan know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @hughshanahan

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v.0.0.1)?
• Authorship: Has the submitting author (@Nikoleta-v3) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @pschloss

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v.0.0.1)?
• Authorship: Has the submitting author (@Nikoleta-v3) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @erahulkulkarni

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v.0.0.1)?
• Authorship: Has the submitting author (@Nikoleta-v3) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @pvanheus

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v.0.0.1)?
• Authorship: Has the submitting author (@Nikoleta-v3) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @fbidu

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release (v.0.0.1)?
• Authorship: Has the submitting author (@Nikoleta-v3) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @hughshanahan, @pschloss it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.1186/s13059-019-1643-1 is OK
- 10.1093/bioinformatics/btw354 is OK
- 10.1186/s13059-019-1891-0 is OK
- 10.1038/nmeth.2066 is OK
- 10.1093/bioinformatics/btv033 is OK
- 10.7717/peerj.7359 is OK
- 10.7717/peerj.1319 is OK
- 10.1093/bioinformatics/bty560 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👉 Check article proof 📄 👈

[emckiernan]

Hi @hughshanahan, @pschloss: Thank you for agreeing to review for JOSE! This is where the action happens: please work your way through the review checklist, feel free to ask questions or post comments here, and also open issues in the submission repository as needed. I'm here to help, so please let me know if you need anything. Thanks!

[emckiernan]

Hi @hughshanahan, @pschloss: I know we're in a difficult time, so I don't want to pressure. But I just wanted to check in with you both about your reviews. How are things going? Anything I can help with?

[hughshanahan]

Hi - aagh. Sorry for dropping the ball on this. I have interviews all day tomorrow and meetings on Wednesday morning. That gives Wednesday afternoon to get after this. Please be very rude to me if I don't get it done then. Sorry again.

[pschloss]

i'll do my best to get it up by the end of the week

[pschloss]

Can @emckiernan confirm that this is the correct review template? It seems to be for software rather than for teaching materials. I was expecting something more like what is given at #78

[pschloss]

Also... what is the preferred way of leaving feedback on the authors' repository? One issue with checkboxes or separate issues for each point?

[emckiernan]

Can @emckiernan confirm that this is the correct review template? It seems to be for software rather than for teaching materials. I was expecting something more like what is given at #78

Hi @pschloss, thanks for your question. I see your point, the review checklist is different from that provided at #78, and is missing sections like the one on pedagogy. This was automatically generated by whedon, so I'm not sure why it is different. Since this is my first time editing for JOSE, I'd like to loop in @labarba, just to make sure we're on the right track. Lorena, do you know why we have a different review template here? Thanks for any help!

[emckiernan]

Also... what is the preferred way of leaving feedback on the authors' repository? One issue with checkboxes or separate issues for each point?

Hi @pschloss: The guidelines say, "Comments in the REVIEW issue should be kept brief, as much as possible, with more lengthy suggestions or requests posted as separate issues, directly in the submission repository." So I'd say, if the comments are short, you can leave something like a checklist, but if the comments are more lengthy, I'd open these as separate issues. You can link back to all of these in your review. Hope that helps!

[labarba]

The checklist is auto-generated on the basis of the submission type chosen by the authors on the submission web form.

@koadman — do you remember choosing the submission type from the drop-down menu back when you submitted?

[labarba]

what is the preferred way of leaving feedback on the authors' repository? One issue with checkboxes or separate issues for each point?

You can decide how you want to structure your feedback in the submission repository (whether to open one or several issues). But be sure to post here a link to the issue with a brief note, to create a cross-link.

Note that this Review thread will be archived together with the paper at publication time, for posterity.

[emckiernan]

Thanks @labarba, we appreciate the help!

[pschloss]

I have posted my review in the authors' repository. I have not completed the checklist above because I'm pretty confident it's the wrong one. I may have additional comments if a different checklist is given.

[kyleniemeyer]

Oh, good point. It looks like whedon posted the JOSS review checklist, rather than the JOSE checklist. @arfon @labarba ?

[arfon]

Oh, good point. It looks like whedon posted the JOSS review checklist, rather than the JOSE checklist. @arfon @labarba ?

Checking...

[arfon]

@kyleniemeyer @labarba - looks like this is marked as software in the JOSE database so I think the review checklist is correct. Were you expecting the checklist for a learning module?

[kyleniemeyer]

@arfon yeah, I think that was a mistake that we didn't catch. This looks to be a learning module instead. Is that an easy fix, or do we just need to manually change the lists?

[arfon]

I'm afraid this is a manual change to the lists so I think you are as well-positioned as I am to make this change :-)

[labarba]

I have manually changed the review checklists. @pschloss — you'll have to go back and re-check items, I'm afraid.

[pschloss]

I've completed the checklist. I think it matches what I have written in my review in the repository.

[emckiernan]

Hi @hughshanahan, just checking in about your review, thanks!

[emckiernan]

Hi @hughshanahan, just checking in about your review, thanks!

Hi @hughshanahan, just checking in again, thanks!

[emckiernan]

Hi @fbidu! You signed up to review for JOSE. Would you like to contribute a review for this submission? The title is "The Gene School: Metagenomics". Please let me know if you're available and willing. Thanks!

[pschloss]

I wonder whether @adina Howe might be interested in reviewing or would have suggestions on others that could provide a review. She works in this area.

[fbidu]

Hi @fbidu! You signed up to review for JOSE. Would you like to contribute a review for this submission? The title is "The Gene School: Metagenomics". Please let me know if you're available and willing. Thanks!

Yes, I am! I think that I have a friend that may be interested as well, can I invite him somehow?

Thanks

[emckiernan]

Yes, I am! I think that I have a friend that may be interested as well, can I invite him somehow?

Thanks

Great, thanks so much @fbidu! I'll add you as a reviewer and then insert a checklist for you above. Could you please send the contact information for your friend to my email emck31 at gmail? I can look it over and invite him if we need an extra reviewer. Thanks again!

[emckiernan]

@whedon add @fbidu as reviewer

[whedon]

OK, @fbidu is now a reviewer

[emckiernan]

Hi @fbidu, I've added a checklist for you at the top of this issue. Please let me know if you have any questions. Thanks!

[pvanheus]

Hi @emckiernan - apologies, I have been ill. I think there is a problem with the template you created though - the URL link points to https://github.com/Nikoleta-v3/Game-Theory-and-Python - which is not the repo being discussed here - at the top of the page https://github.com/thegeneschool/metagenomics is mentioned. What is going on?

[emckiernan]

Hi @emckiernan - apologies, I have been ill. I think there is a problem with the template you created though - the URL link points to https://github.com/Nikoleta-v3/Game-Theory-and-Python - which is not the repo being discussed here - at the top of the page https://github.com/thegeneschool/metagenomics is mentioned. What is going on?

Hi @pvanheus, no worries at all, I'm sorry to hear you were ill. I hope you're feeling better! And thank you for flagging the URL error. I'm not sure why the checklist originally generated by the bot linked to that repo. After that it was copy-paste errors when checklists were manually added. But you're right, the correct repository for review is https://github.com/thegeneschool/metagenomics. I can correct the links above.

[emckiernan]

Hi @emckiernan - apologies, I have been ill. I think there is a problem with the template you created though - the URL link points to https://github.com/Nikoleta-v3/Game-Theory-and-Python - which is not the repo being discussed here - at the top of the page https://github.com/thegeneschool/metagenomics is mentioned. What is going on?

Hi @pvanheus, no worries at all, I'm sorry to hear you were ill. I hope you're feeling better! And thank you for flagging the URL error. I'm not sure why the checklist originally generated by the bot linked to that repo. After that it was copy-paste errors when checklists were manually added. But you're right, the correct repository for review is https://github.com/thegeneschool/metagenomics. I can correct the links above.

I see now! I think this was a copy-paste error early on when we changed the format of the checklist. All the links should point to the correct repository now. Thanks, @pvanheus!

Tagging @fbidu in here, please check the repo link in your checklist to be sure all comments/issues are made on the correct repository. Thanks!

[emckiernan]

Hi @pvanheus and @fbidu, hope you're both doing well! Just wanted to check in about your reviews. Anything I can help with?

[emckiernan]

Hi @pvanheus and @fbidu, just checking in again! Any questions or areas I can help?

[pvanheus]

@emckiernan I have submitted a much delayed review. I must admit that reviewing this material was not easy, but working through your checklist and considering the material from the perspective of someone who might want to adopt it helped in the end. I'll also note that I opened two issues (here and here) that have not been addressed by the authors since October.

[emckiernan]

@pschloss and @pvanheus, thank you so much for your reviews, we really appreciate your hard work on this!
@koadman, we now have two reviews, yay! Do you have any questions for the reviewers, or for me, on how to move forward with revisions?

[koadman]

Thanks all, yes ready to move forward with revisions in the new year. Plenty to do here, thanks for the helpful feedback!

[labarba]

Hi @koadman 👋 — do let us know how you are getting along with revising your submission!
We are all digging out of the rubble of piled up work, and overall world chaos, so just give as a quick update when you can.

[labarba]

hi everyone 👋 — @emckiernan as editor, and @koadman as author — what do you think we should do here? This submission is stalled mid-review for many months. Is there a will to bring it to the finish line? Please let us know!

[labarba]

Hi folks! Since we've had no reply from the authors for some time, I'm going to go ahead and pause this review.

@koadman — Do let us know here if you would like to restart the review.

[fbidu]

@labarba oh my, this absolutely fell off of my radar, I'm sorry. I can review this on Saturday. Do you think it is still worth it?

Thanks!

[emckiernan]

Hi @fbidu, no worries, and thank you for your offer! We did get two reviewers in the end, so I think we're ok for now. However, if we need additional feedback at some point, I will definitely circle back to you. Thanks again!