[REVIEW]: Synthia: multi-dimensional synthetic data generation in Python

[whedon]

Submitting author: @dmey (D. Meyer)
Repository: https://github.com/dmey/synthia
Version: v1.1.0
Editor: @oliviaguest
Reviewer: @khinsen, @mnarayan
Archive: 10.5281/zenodo.5358432

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b"><img src="https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b/status.svg)](https://joss.theoj.org/papers/e199a37aa903382a8ba34a70823ed72b)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@khinsen & @mnarayan, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @oliviaguest know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @khinsen

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@dmey) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @mnarayan

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@dmey) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @khinsen, @mnarayan it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #2863 with the following error:

Can't find any papers to compile :-(

[dmey]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[oliviaguest]

Hi all! 👋 Thank you so much, @khinsen, @mnarayan for accepting to review this. Please read the instructions above. Any questions, feedback on the paper, etc., please post here. Any very code-specific questions, suggestions, etc., please use the issues in the code repo and link to them from this thread so we can all keep track of them. 🌸

For an example of how this process plays out feel free to skim previous reviews, such as: #2285 and #2348. ☺️

[oliviaguest]

@whedon check references from branch joss-paper

[whedon]

Attempting to check references... from custom branch joss-paper

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1201/b17116 is OK
- 10.1109/DSAA.2016.49 is OK
- 10.13140/2.1.4476.8963 is OK
- 10.1109/ICDE.2008.4497436 is OK
- 10.5334/jors.148 is OK
- 10.5281/zenodo.4287554 is OK
- 10.5281/zenodo.4288292 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[khinsen]

Question to @dmey, @tnagler, @letmaik: I am investigating authorship as required by JOSS reviewing guidelines. There is no doubt that @dmey is the main contributor to this project. Of the two other contributors, whose contributions look roughly equal in importance from the "size of commits" point of view, @tnagler is a co-author whereas @letmaik is merely acknowledged for "comments and contributions". If that's OK with all of you, it's fine with me as well of course, but it looks surprising.

[letmaik]

@khinsen That's fine, we discussed this offline in advance, no problem there.

[dmey]

@khinsen it's good that you picked this up as it may appear a bit confusing -- all commits during development were squashed therefore the current metadata is not very indicative of the amount of work and contributions made during development. @tnagler made substantial contributions in the conceptual and practical development of the tool. @letmaik gave us suggestions and made contributions to the project.

[whedon]

👋 @khinsen, please update us on how your review is going.

[whedon]

👋 @mnarayan, please update us on how your review is going.

[khinsen]

Thanks @dmey and @letmaik! I realize that GitHub statistics share the problem of bibliometry in being superficial.

[khinsen]

@whedon Going fine, thanks for asking!

[whedon]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

[khinsen]

OK, I am being mean in talking like that to a bot, but I really don't know what I am expected to do in reply to the request for "updating us"!

[oliviaguest]

@khinsen the bot is just checking to see you are indeed doing the review, exactly like a reminder from a more "traditional" journal in your email inbox. Thus, you are able to ignore as you're already doing it.

If you are curious what commands @whedon does accept, I am sure you can figure that out too — haha: https://joss.readthedocs.io/en/latest/whedon.html

[khinsen]

@oliviaguest The checklist asks "Do references in the text use the proper citation syntax?", referring to a description of Markdown syntax for citations. What I have for review is a PDF file. I don't quite understand what I am supposed to do. Should I be looking at some Markdown source instead? If so, where can I find it?

[khinsen]

@dmey After going through the checklist for the software, here comes my review of the paper itself. Overall, it looks very good, there is just one point I would like you to address:

"Do the authors describe how this software compares to other commonly-used packages?"

No. Please add a statement on this question, even if it is only "we are not aware of any other comparable software". The Wikipedia page on Synthetic Data lists one Open Source package (DataGenerator) and one publicly available package without a licence (Dataset Generator), in addition to proprietary software.

[oliviaguest]

@khinsen do you mean where is the paper being compiled from? Here: https://github.com/dmey/synthia/tree/joss-paper

[khinsen]

@oliviaguest Exactly. Thanks for the pointer, I hadn't considered looking for a specific branch. Perhaps the review instructions should contain that pointer, given that reviewers are expected to check the Markdown source.

[oliviaguest]

@khinsen I thought (wrongly!) it was obvious when we ask for @whedon to do this:

@whedon generate pdf from branch joss-paper

But either way — it's no problem at all and I am more than happy to explain it (there is so much to process for those who have never used JOSS before anyway). ☺️

[khinsen]

@oliviaguest Everything is obvious if you do it often enough! Authors and editors are a lot more familiar with @whedon than reviewers, and for now that has been my only role in JOSS. We can usually do our jobs without ever talking to @whedon. Which I think is great, reviewing for JOSS is a real pleasure because of the absence of technical boilerplate tasks.

[dmey]

@danielskatz thanks -- sounds good with me.

[dmey]

@danielskatz and @oliviaguest would you have an update about this submission? Please let me know if you are waiting for further changes from my side. If not please can you use #2863 (comment) as final version for the summary-paper and version 1.1.0 with corresponding Zenodo archive https://zenodo.org/record/5358432 (currently set to an older version).

[oliviaguest]

@whedon set v1.1.0 as version

[whedon]

OK. v1.1.0 is the version.

[oliviaguest]

@whedon set 10.5281/zenodo.5358432 as archive

[whedon]

OK. 10.5281/zenodo.5358432 is the archive.

[oliviaguest]

@whedon recommend-accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3390/s19051181 is OK
- 10.1038/s41586-020-2649-2 is OK
- 10.5334/jors.148 is OK
- 10.1201/b17116 is OK
- 10.1109/ICDMW51313.2020.00082 is OK
- 10.5194/gmd-14-5205-2021 is OK
- 10.5281/zenodo.4287554 is OK
- 10.5281/zenodo.4288292 is OK
- 10.18637/jss.v074.i11 is OK
- 10.1109/DSAA.2016.49 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- None

INVALID DOIs

- 10.1029/2021MS002550 is INVALID

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#2602

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#2602, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[oliviaguest]

@dmey can you check that invalid DOI please and the final proof?

[kthyng]

I see the version and archive are finished and attached, and I read through the paper and it looks good to me. Ready when we hear back from @oliviaguest's questions.

[dmey]

@oliviaguest and @kthyng I am happy with the final proof, thank you. Re DOI showing as invalid this is correct but it has not yet been accepted (and deposited). The DOI will not change so I can (a) leave it as is, (b) update the reference to include 'under review', or (c) wait for the DOI to be deposited -- this should take a few weeks. Happy with any of those options.

[kthyng]

I'm fine with letting the DOI stand. What do you think @oliviaguest?

[kthyng]

I think we can proceed without hearing from @oliviaguest on this small issue. Seems fine as is.

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02863 joss-papers#2608
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.02863
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kthyng]

Congrats on your new publication @dmey! Many thanks to editor @oliviaguest and reviewers @khinsen and @mnarayan for your time, hard work, and expertise!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02863/status.svg)](https://doi.org/10.21105/joss.02863)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02863">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02863/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02863/status.svg
   :target: https://doi.org/10.21105/joss.02863

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[dmey]

Thank you @kthyng and many thanks again to @oliviaguest @danielskatz @khinsen and @mnarayan for your help and feedback provided.

[oliviaguest]

@dmey you are welcome and glad to see this out! ✨