Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW] Detection and attribution of climate change: A deep learning and variational approach #173

Closed
44 of 63 tasks
acocac opened this issue May 24, 2023 · 13 comments · Fixed by #186
Closed
44 of 63 tasks

[REVIEW] Detection and attribution of climate change: A deep learning and variational approach #173

acocac opened this issue May 24, 2023 · 13 comments · Fixed by #186
Assignees
@annefou @asthanameghna @dbhatedin @NHomer-Edi
Labels
ci2023-reproducibility-challenge notebook Add/update notebook

Comments

@acocac
Copy link
Member

acocac commented May 24, 2023 •
edited
Loading

Notebook Review: Issue #171

Binder

Submitting author: @ancazugo @SkirOwen @ViktorDomazetoski

Repository: https://github.com/eds-book-gallery/CI2023-RC-team2

Paper: https://doi.org/10.1017/eds.2022.17

Editor: @ampersandmcd @annefou

Reviewer: @asthanameghna @NHomer-Edi @dbhatedin

Managing EiC: @acocac

Status

Reviewer instructions & questions

Hi 👋 @asthanameghna @NHomer-Edi @dbhatedin, please carry out your review in this issue by updating the checklist below.

As a reviewer, you contribute to the technical quality of the content published by our community.

Before the review, EiC checked if the submission fits the minimum requirements.

The quality of the proposed contribution can be assessed through scientific, technical and code criteria.

The reviewer guidelines are available here: https://edsbook.org/publishing/guidelines/guidelines-reviewers.html.
Any questions/concerns please let @ampersandmcd know.

Review checklist for @asthanameghna

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.

Conflict of interest

  • As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Code of conduct an peer-review principles

General checks

  • Notebook: Is the notebook file (notebook.ipynb) part of the notebook repository?
  • Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
  • Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?

Reproducibility

  • Does the notebook run in a local environment?
  • Does the notebook build and run in binder?
  • Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?

Pedagogy

  • Are the notebook purpose and highlights clear?
  • Does the notebook demonstrate some specific data analysis or visualisation techniques?
  • Is the notebook well documented, using both markdown cells and comments in code cells?
  • Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
  • Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?

Ethical

  • Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
  • Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
  • Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?

Other Requirements

  • All mentioned software should be formally and consistently cited wherever possible.
  • Acronyms should be spelled out upon first mention.
  • License conditions on images and figures must be respected (Creative Commons, etc.).

Final approval (post-review)

  • Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.

Review checklist for @NHomer-Edi

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.

Conflict of interest

  • As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Code of conduct an peer-review principles

General checks

  • Notebook: Is the notebook file (notebook.ipynb) part of the notebook repository?
  • Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
  • Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?

Reproducibility

  • Does the notebook run in a local environment?
  • Does the notebook build and run in binder?
  • Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?

Pedagogy

  • Are the notebook purpose and highlights clear?
  • Does the notebook demonstrate some specific data analysis or visualisation techniques?
  • Is the notebook well documented, using both markdown cells and comments in code cells?
  • Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
  • Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?

Ethical

  • Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
  • Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
  • Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?

Other Requirements

  • All mentioned software should be formally and consistently cited wherever possible.
  • Acronyms should be spelled out upon first mention.
  • License conditions on images and figures must be respected (Creative Commons, etc.).

Final approval (post-review)

  • Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.

Review checklist for @dbhatedin

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide.

Conflict of interest

  • As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Code of conduct an peer-review principles

General checks

  • Notebook: Is the notebook file (notebook.ipynb) part of the notebook repository?
  • Contribution and authorship: Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
  • Scope and eligibility: Does the submission contain an original and complete analysis according to the scope of EDS book?

Reproducibility

  • Does the notebook run in a local environment?
  • Does the notebook build and run in binder?
  • Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?

Pedagogy

  • Are the notebook purpose and highlights clear?
  • Does the notebook demonstrate some specific data analysis or visualisation techniques?
  • Is the notebook well documented, using both markdown cells and comments in code cells?
  • Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
  • Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?

Ethical

  • Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
  • Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
  • Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?

Other Requirements

  • All mentioned software should be formally and consistently cited wherever possible.
  • Acronyms should be spelled out upon first mention.
  • License conditions on images and figures must be respected (Creative Commons, etc.).

Final approval (post-review)

  • Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.

Additional instructions

Reviewer general comments are welcome on this REVIEW issue or directly to the notebook repository.

If you do the latter, you will find a Pull Request titled REVIEW where you can carry out the discussion with authors through ReviewNB, a third-party plugin in GitHub for displaying and commenting Jupyter Notebooks (see further details here).

In addition to ReviewNB, we suggest to explore or run the notebook in:

  • Binder (run): Click the Launch Binder button at the top level of this message.
@acocac
Copy link
Member Author

acocac commented May 24, 2023

The report below counts blank lines, comment lines, and physical lines of source code files using cloc. It was generated according to the latest commit 0b0ee44 of the review branch from the target repository.

Reviewers and authors feel free this info only for informative purposes. We will generate a similar report after the review process.

Software report (experimental):

github.com/AlDanial/cloc v 1.97  T=4.23 s (5.7 files/s, 2060.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Jupyter Notebook                 5              0           3549           2082
Python                          10            713            260           1669
YAML                             7             20             23            339
Markdown                         1             11              0             34
JSON                             1              0              0              5
-------------------------------------------------------------------------------
SUM:                            24            744           3832           4129
-------------------------------------------------------------------------------

@acocac
Copy link
Member Author

acocac commented May 24, 2023

👋 @asthanameghna @NHomer-Edi @dbhatedin we will conduct the review in this issue.

Please read through the above information and let me know if you have any questions about the review process.

Thank you 🙏

@acocac
Copy link
Member Author

acocac commented May 26, 2023

Hi @asthanameghna @NHomer-Edi @dbhatedin! How are you doing? Is everything OK with the review? Please let us know here when it's ready, then we'll suggest authors to go through each of your comments.

@NHomer-Edi
Copy link
Collaborator

Hi @asthanameghna @NHomer-Edi @dbhatedin! How are you doing? Is everything OK with the review? Please let us know here when it's ready, then we'll suggest authors to go through each of your comments.

Hi @acocac, my comments are ready to be sent to the authors :)

@asthanameghna
Copy link
Collaborator

My comment are ready too. I agree with all the comments by @NHomer-Edi. I am happy for the particiants to see the comments.

@dbhatedin
Copy link
Collaborator

Hi reviewer team! Happy to share with the participants, my comments are here: "Its a very in-depth work with several innovative constructs in AI being applied at critical points being the main strength of this work. My comments are more towards the layout of the available materials - like finally the data is put up in GitLab (and code as well but thats there in github as well) which could be seen in data fetch sub-section and from the published paper's ending list. Would be great to have the data in someplace like Zenodo or OpenAire.eu. So the published paper becomes very important, a primary go to for anybody wanting to know the data being run in the code. It would be very helpful if we can see some comments in the code as well pointing to the charts, graphs and their labels. Currently some graphs are appearing towards the bottom of the notebook run without any comments, and one has to compare with the paper to properly understand. One last comment is to state the impact in a few lines, as to make a statement on what would be missing if such AI techniques were not used. In the conclusion section in the paper future work has been mentioned but the impact part would be good to add in some section."

@acocac
Copy link
Member Author

acocac commented May 26, 2023

@NHomer-Edi @asthanameghna @dbhatedin thanks for going through the checklist and provide feedback of the notebook repository.

@ancazugo @SkirOwen @ViktorDomazetoski please go through reviewers' comments and address their suggestions if relevant to have the updated version of the notebook. Note the specific comments are available in the PR of the notebook repository (see here).

@acocac
Copy link
Member Author

acocac commented Jun 20, 2023

@ancazugo @SkirOwen @ViktorDomazetoski may I ask your attention on above general comments left by reviewers? A reminder you can find their specific comments here.

Please let me know if you have any questions, I'm happy to help.

@annefou
Copy link
Collaborator

annefou commented Aug 17, 2023

@ancazugo @SkirOwen @ViktorDomazetoski I just went through the reviewer's comments and see that a few of them are still pending. The notebook is really great and it would be really nice to publish it as soon as possible. Any plans to finalise it? I am happy to help if needed. Thanks,

@ViktorDomazetoski
Copy link

ViktorDomazetoski commented Aug 18, 2023 via email

@annefou
Copy link
Collaborator

annefou commented Aug 18, 2023

@ViktorDomazetoski You did a very good job! Sorry I forgot I had to check another branch for the notebook. I went through all the comments and check the updated notebook. You implemented all the requested changes. It is great! There is a comment on plots that could be replaced by interactive plots: I think we can keep the plots as they are and think about adding interactivity in a future version of the notebook (which could be done by anyone interested by interactive visualisation).

From my side, the notebook is ready to be published. Thanks again for your hard work!

@acocac acocac mentioned this issue Aug 24, 2023
Merged
22 tasks
@acocac
Copy link
Member Author

acocac commented Sep 11, 2023

👋 @ancazugo @SkirOwen @ViktorDomazetoski

Congratulations! Your notebook submitted to 2023 CI Reproducibility Challenge is published! 🎉 We'll announce the release among EDS book social networks between today Monday 11th and tomorrow Tuesday 12th September.

Finally, big thanks to our editor: @annefou and the reviewers: @asthanameghna @NHomer-Edi @dbhatedin — your inputs were valuable to improving the quality of the submission 🙏

@ViktorDomazetoski
Copy link

ViktorDomazetoski commented Sep 11, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@annefou annefou

@asthanameghna asthanameghna

@dbhatedin dbhatedin

@NHomer-Edi NHomer-Edi

Labels
ci2023-reproducibility-challenge notebook Add/update notebook
Projects
None yet
Milestone
No milestone
6 participants
@annefou @acocac @asthanameghna @dbhatedin @NHomer-Edi @ViktorDomazetoski