From 6b5ed03e5716d320d2a6f98178c41727b07ba6c2 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 26 Apr 2024 11:36:47 -0400 Subject: [PATCH 01/31] stub out Contributor Guide #10531 --- doc/sphinx-guides/source/contributor/index.md | 5 ++ doc/sphinx-guides/source/contributor/intro.md | 70 +++++++++++++++++++ doc/sphinx-guides/source/index.rst | 1 + 3 files changed, 76 insertions(+) create mode 100644 doc/sphinx-guides/source/contributor/index.md create mode 100644 doc/sphinx-guides/source/contributor/intro.md diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md new file mode 100644 index 00000000000..8f49785780b --- /dev/null +++ b/doc/sphinx-guides/source/contributor/index.md @@ -0,0 +1,5 @@ +# Contributor Guide + +```{toctree} +intro.md +``` diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md new file mode 100644 index 00000000000..08de9c642a2 --- /dev/null +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -0,0 +1,70 @@ +# Introduction + +Please note: this is a copy/paste from but we intend to split the content on this intro page into sub-pages. + +Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. You don't need permission to participate. Just jump in. If you have questions, please reach out using one or more of the channels described below. + +We aren't just looking for developers. There are many ways to contribute to Dataverse. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! + +```{contents} Contents: +:local: +:depth: 3 +``` + +## Ideas/Feature Requests + +Your idea or feature request might already be captured in the Dataverse [issue tracker] on GitHub but if not, the best way to bring it to the community's attention is by posting on the [dataverse-community Google Group][] or bringing it up on a [Community Call][]. You're also welcome to make some noise in [chat.dataverse.org][] or cram your idea into 280 characters and mention [@dataverseorg][] on Twitter. To discuss your idea privately, please email it to + +There's a chance your idea is already on our roadmap, which is available at + +[chat.dataverse.org]: http://chat.dataverse.org +[issue tracker]: https://github.com/IQSS/dataverse/issues +[@dataverseorg]: https://twitter.com/dataverseorg + +## Usability Testing + +Please email us at if you are interested in participating in usability testing. + +## Bug Reports/Issues + +An issue is a bug (a feature is no longer behaving the way it should) or a feature (something new to Dataverse that helps users complete tasks). You can browse the Dataverse [issue tracker] on GitHub by open or closed issues or by milestones. + +Before submitting an issue, please search the existing issues by using the search bar at the top of the page. If there is an existing open issue that matches the issue you want to report, please add a comment to it. + +If there is no pre-existing issue or it has been closed, please click on the "New Issue" button, log in, and write in what the issue is (unless it is a security issue which should be reported privately to security@dataverse.org). + +If you do not receive a reply to your new issue or comment in a timely manner, please email with a link to the issue. + +### Writing an Issue + +For the subject of an issue, please start it by writing the feature or functionality it relates to, i.e. "Create Account:..." or "Dataset Page:...". In the body of the issue, please outline the issue you are reporting with as much detail as possible. In order for the Dataverse development team to best respond to the issue, we need as much information about the issue as you can provide. Include steps to reproduce bugs. Indicate which version you're using, which is shown at the bottom of the page. We love screenshots! + +### Issue Attachments + +You can attach certain files (images, screenshots, logs, etc.) by dragging and dropping, selecting them, or pasting from the clipboard. Files must be one of GitHub's [supported attachment formats] such as png, gif, jpg, txt, pdf, zip, etc. (Pro tip: A file ending in .log can be renamed to .txt so you can upload it.) If there's no easy way to attach your file, please include a URL that points to the file in question. + +[supported attachment formats]: https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/ + +## Documentation + +The source for the documentation is in the GitHub repo under the "[doc][]" folder. If you find a typo or inaccuracy or something to clarify, please send us a pull request! For more on the tools used to write docs, please see the {doc}`/developers/documentation` section of the Developer Guide. + +[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source + +## Code/Pull Requests + +We love code contributions. Developers are not limited to the main Dataverse code in this git repo. You can help with API client libraries in your favorite language that are mentioned in the {doc}`/api/index` or create a new library. You can help work on configuration management code that's mentioned in the {doc}`/installation/index` . The Installation Guide also covers a relatively new concept called "external tools" that allows developers to create their own tools that are available from within an installation of Dataverse. + +If you are interested in working on the main Dataverse code, great! Before you start coding, please reach out to us either on the [dataverse-community Google Group][], the [dataverse-dev Google Group][], [chat.dataverse.org][], or via to make sure the effort is well coordinated and we avoid merge conflicts. We maintain a list of [community contributors][] and [dev efforts][] the community is working on so please let us know if you'd like to be added or removed from either list. + +Please read {doc}`/developers/version-control` in the Developer Guide to understand how we use the "git flow" model of development and how we will encourage you to create a GitHub issue (if it doesn't exist already) to associate with your pull request. That page also includes tips on making a pull request. + +After making your pull request, your goal should be to help it advance through our kanban board at . If no one has moved your pull request to the code review column in a timely manner, please reach out. Note that once a pull request is created for an issue, we'll remove the issue from the board so that we only track one card (the pull request). + +Thanks for your contribution! + +[dataverse-community Google Group]: https://groups.google.com/group/dataverse-community +[Community Call]: https://dataverse.org/community-calls +[dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev +[community contributors]: https://docs.google.com/spreadsheets/d/1o9DD-MQ0WkrYaEFTD5rF_NtyL8aUISgURsAXSL7Budk/edit?usp=sharing +[dev efforts]: https://github.com/orgs/IQSS/projects/34/views/6 diff --git a/doc/sphinx-guides/source/index.rst b/doc/sphinx-guides/source/index.rst index 3184160b387..af394108eea 100755 --- a/doc/sphinx-guides/source/index.rst +++ b/doc/sphinx-guides/source/index.rst @@ -17,6 +17,7 @@ These documentation guides are for the |version| version of Dataverse. To find g admin/index api/index installation/index + contributor/index developers/index container/index style/index From 8496c10868a83ff8074444e59941f1a43c47a701 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 10 May 2024 10:23:11 -0400 Subject: [PATCH 02/31] rewrite ideas/features --- doc/sphinx-guides/source/contributor/intro.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 08de9c642a2..951f2ec49bb 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -13,13 +13,14 @@ We aren't just looking for developers. There are many ways to contribute to Data ## Ideas/Feature Requests -Your idea or feature request might already be captured in the Dataverse [issue tracker] on GitHub but if not, the best way to bring it to the community's attention is by posting on the [dataverse-community Google Group][] or bringing it up on a [Community Call][]. You're also welcome to make some noise in [chat.dataverse.org][] or cram your idea into 280 characters and mention [@dataverseorg][] on Twitter. To discuss your idea privately, please email it to +1. Please check if your idea or feature request is already captured in our [issue tracker][] or [roadmap][]. +1. Bring your idea to the community by posting on our [Google Group][] or [chat.dataverse.org][]. +1. To discuss privately, email us at support@dataverse.org. -There's a chance your idea is already on our roadmap, which is available at - -[chat.dataverse.org]: http://chat.dataverse.org [issue tracker]: https://github.com/IQSS/dataverse/issues -[@dataverseorg]: https://twitter.com/dataverseorg +[roadmap]: https://www.iq.harvard.edu/roadmap-dataverse-project +[chat.dataverse.org]: http://chat.dataverse.org +[Google Group]: https://groups.google.com/group/dataverse-community ## Usability Testing From 6382517eddd22421164009b292df760ea7f1fd20 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 10 May 2024 10:29:57 -0400 Subject: [PATCH 03/31] link to usability testing form, move to bottom --- doc/sphinx-guides/source/contributor/intro.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 951f2ec49bb..4d0cc86a7ea 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -22,10 +22,6 @@ We aren't just looking for developers. There are many ways to contribute to Data [chat.dataverse.org]: http://chat.dataverse.org [Google Group]: https://groups.google.com/group/dataverse-community -## Usability Testing - -Please email us at if you are interested in participating in usability testing. - ## Bug Reports/Issues An issue is a bug (a feature is no longer behaving the way it should) or a feature (something new to Dataverse that helps users complete tasks). You can browse the Dataverse [issue tracker] on GitHub by open or closed issues or by milestones. @@ -64,6 +60,12 @@ After making your pull request, your goal should be to help it advance through o Thanks for your contribution! +## Usability Testing + +Please email us at or fill in our [feedback form][] if you are interested in participating in usability testing. + +[feedback form]: https://goo.gl/forms/p7uu3GfiWYSlJrsi1 + [dataverse-community Google Group]: https://groups.google.com/group/dataverse-community [Community Call]: https://dataverse.org/community-calls [dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev From 9747685a9c44aeb85917066c43e319ca661f473c Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 10 May 2024 10:49:00 -0400 Subject: [PATCH 04/31] love your ideas, add support link --- doc/sphinx-guides/source/contributor/intro.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 4d0cc86a7ea..5ce0afedff6 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -13,9 +13,11 @@ We aren't just looking for developers. There are many ways to contribute to Data ## Ideas/Feature Requests +We would love to hear your ideas!💡 + 1. Please check if your idea or feature request is already captured in our [issue tracker][] or [roadmap][]. 1. Bring your idea to the community by posting on our [Google Group][] or [chat.dataverse.org][]. -1. To discuss privately, email us at support@dataverse.org. +1. To discuss privately, email us at . [issue tracker]: https://github.com/IQSS/dataverse/issues [roadmap]: https://www.iq.harvard.edu/roadmap-dataverse-project From 5b813568b0207896f81d699850b38b9cbf042058 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 10 May 2024 11:04:00 -0400 Subject: [PATCH 05/31] simplify bug report section --- doc/sphinx-guides/source/contributor/intro.md | 20 ++++--------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 5ce0afedff6..75fd93e9914 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -24,25 +24,13 @@ We would love to hear your ideas!💡 [chat.dataverse.org]: http://chat.dataverse.org [Google Group]: https://groups.google.com/group/dataverse-community -## Bug Reports/Issues +## Bug Reports -An issue is a bug (a feature is no longer behaving the way it should) or a feature (something new to Dataverse that helps users complete tasks). You can browse the Dataverse [issue tracker] on GitHub by open or closed issues or by milestones. +Before submitting an issue, please search for existing issues in our [issue tracker][]. If there is an existing open issue that matches the issue you want to report, please add a comment to it or give it a 👍. -Before submitting an issue, please search the existing issues by using the search bar at the top of the page. If there is an existing open issue that matches the issue you want to report, please add a comment to it. +If there is no pre-existing issue or it has been closed, please open a new issue (unless it is a security issue which should be reported privately to ). -If there is no pre-existing issue or it has been closed, please click on the "New Issue" button, log in, and write in what the issue is (unless it is a security issue which should be reported privately to security@dataverse.org). - -If you do not receive a reply to your new issue or comment in a timely manner, please email with a link to the issue. - -### Writing an Issue - -For the subject of an issue, please start it by writing the feature or functionality it relates to, i.e. "Create Account:..." or "Dataset Page:...". In the body of the issue, please outline the issue you are reporting with as much detail as possible. In order for the Dataverse development team to best respond to the issue, we need as much information about the issue as you can provide. Include steps to reproduce bugs. Indicate which version you're using, which is shown at the bottom of the page. We love screenshots! - -### Issue Attachments - -You can attach certain files (images, screenshots, logs, etc.) by dragging and dropping, selecting them, or pasting from the clipboard. Files must be one of GitHub's [supported attachment formats] such as png, gif, jpg, txt, pdf, zip, etc. (Pro tip: A file ending in .log can be renamed to .txt so you can upload it.) If there's no easy way to attach your file, please include a URL that points to the file in question. - -[supported attachment formats]: https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/ +If you do not receive a reply to your new issue or comment in a timely manner, please ping us at [chat.dataverse.org][]. ## Documentation From 4eede606ebabddc71705a76915fff0cc774f790b Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 10 May 2024 11:27:53 -0400 Subject: [PATCH 06/31] link to "reporting security issues" section --- doc/sphinx-guides/source/contributor/intro.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 75fd93e9914..5348f4b307f 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -28,7 +28,7 @@ We would love to hear your ideas!💡 Before submitting an issue, please search for existing issues in our [issue tracker][]. If there is an existing open issue that matches the issue you want to report, please add a comment to it or give it a 👍. -If there is no pre-existing issue or it has been closed, please open a new issue (unless it is a security issue which should be reported privately to ). +If there is no pre-existing issue or it has been closed, please open a new issue (unless it is a security issue which should be reported privately to as discussed under {ref}`reporting-security-issues` in the Installation Guide). If you do not receive a reply to your new issue or comment in a timely manner, please ping us at [chat.dataverse.org][]. From 18c5b9b11936027e1149a6029f8b261a81947303 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 7 Jun 2024 10:20:14 -0400 Subject: [PATCH 07/31] Update intro.md --- doc/sphinx-guides/source/contributor/intro.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md index 5348f4b307f..47111d0a663 100644 --- a/doc/sphinx-guides/source/contributor/intro.md +++ b/doc/sphinx-guides/source/contributor/intro.md @@ -2,9 +2,9 @@ Please note: this is a copy/paste from but we intend to split the content on this intro page into sub-pages. -Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. You don't need permission to participate. Just jump in. If you have questions, please reach out using one or more of the channels described below. +Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. We aren't just looking for developers. There are many ways to contribute to Dataverse. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! -We aren't just looking for developers. There are many ways to contribute to Dataverse. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! +If you have questions, please reach out using one or more of the channels described below. ```{contents} Contents: :local: From 32be795562b821449bdff0076ccba2860eb23e42 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 7 Jun 2024 11:56:59 -0400 Subject: [PATCH 08/31] remove intro page and put its content in index and code pages #10531 This is a bit of an experiment discussed with @vaidap and @DieuwertjeBloemen where we are trying to put more content on the index page rather than a table of contents. Because we have a subpage (code.md) we still need to put it in a toctree (or it will be an orphan), so I hid it the best I could within the flow of the text. --- doc/sphinx-guides/source/contributor/code.md | 26 ++++++++ doc/sphinx-guides/source/contributor/index.md | 55 +++++++++++++++- doc/sphinx-guides/source/contributor/intro.md | 63 ------------------- .../source/developers/documentation.rst | 9 +++ 4 files changed, 89 insertions(+), 64 deletions(-) create mode 100644 doc/sphinx-guides/source/contributor/code.md delete mode 100644 doc/sphinx-guides/source/contributor/intro.md diff --git a/doc/sphinx-guides/source/contributor/code.md b/doc/sphinx-guides/source/contributor/code.md new file mode 100644 index 00000000000..941ce1fd6bc --- /dev/null +++ b/doc/sphinx-guides/source/contributor/code.md @@ -0,0 +1,26 @@ +# Contributing Code + +```{contents} Contents: +:local: +:depth: 3 +``` + +## Welcome! + +We love code contributions. Developers are not limited to the main Dataverse code in this git repo. You can help with API {doc}`/api/client-libraries` in your favorite language or create a new library. You can help work on configuration management code that's mentioned in the {doc}`/installation/index`. See {doc}`/api/external-tools` for how you can create your own tool that is available from within an installation of Dataverse. + +If you are interested in working on the main Dataverse code, great! Before you start coding, please reach out to us either on the [dataverse-community Google Group][], the [dataverse-dev Google Group][], [chat.dataverse.org][], or via to make sure the effort is well coordinated and we avoid merge conflicts. We maintain a list of [community contributors][] and [dev efforts][] the community is working on so please let us know if you'd like to be added or removed from either list. + +## Making a Pull Request + +Please read {doc}`/developers/version-control` in the Developer Guide to understand how we use the "git flow" model of development and how we will encourage you to create a GitHub issue (if it doesn't exist already) to associate with your pull request. That page also includes tips on making a pull request. + +After making your pull request, your goal should be to help it advance through our kanban board at . If no one has moved your pull request to the code review column in a timely manner, please reach out. Note that once a pull request is created for an issue, we'll remove the issue from the board so that we only track one card (the pull request). + +Thanks for your contribution! + +[dataverse-community Google Group]: https://groups.google.com/group/dataverse-community +[dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev +[chat.dataverse.org]: http://chat.dataverse.org +[community contributors]: https://docs.google.com/spreadsheets/d/1o9DD-MQ0WkrYaEFTD5rF_NtyL8aUISgURsAXSL7Budk/edit?usp=sharing +[dev efforts]: https://github.com/orgs/IQSS/projects/34/views/6 diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 8f49785780b..7a965b994e1 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -1,5 +1,58 @@ # Contributor Guide +Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! + +```{contents} Contents: +:local: +:depth: 3 +``` + +## Ideas and Feature Requests + +We would love to hear your ideas!💡 + +1. Please check if your idea or feature request is already captured in our [issue tracker][] or [roadmap][]. +1. Bring your idea to the community by posting on our [Google Group][] or [chat.dataverse.org][]. +1. To discuss privately, email us at . + +[issue tracker]: https://github.com/IQSS/dataverse/issues +[roadmap]: https://www.iq.harvard.edu/roadmap-dataverse-project +[chat.dataverse.org]: http://chat.dataverse.org +[Google Group]: https://groups.google.com/group/dataverse-community + +## Bug Reports + +Before submitting an issue, please search for existing issues in our [issue tracker][]. If there is an existing open issue that matches the issue you want to report, please add a comment to it or give it a 👍 (thumbs up). + +If there is no pre-existing issue or it has been closed, please open a new issue (unless it is a security issue which should be reported privately to as discussed under {ref}`reporting-security-issues` in the Installation Guide). + +If you do not receive a reply to your new issue or comment in a timely manner, please ping us at [chat.dataverse.org][]. + +## Documentation + +The source for the documentation is in the GitHub repo under the "[doc][]" folder. If you find a typo or inaccuracy or something to clarify, please send us a pull request! For more on the tools used to write docs, please see the {doc}`/developers/documentation` section of the Developer Guide. + +Please observe the following when writing documentation: + +- Use American English spelling. +- Use examples when possible. +- Break up longer paragraphs. +- Use "double quotes" instead of 'single quotes'. +- Favor "and" (data and code) over slashes (data/code). + +[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source + +## Code and Pull Requests + +Dataverse is open source and we love code contributions. Developers are not limited to the main Dataverse code in this git repo. We have projects in C, C++, Go, Java, Javascript, Julia, PHP, Python, R, Ruby, TypeScript and more. To get started, please see the following pages: + ```{toctree} -intro.md +:maxdepth: 1 +code.md ``` + +## Usability Testing + +Please email us at or fill in our [feedback form][] if you are interested in participating in usability testing. + +[feedback form]: https://goo.gl/forms/p7uu3GfiWYSlJrsi1 diff --git a/doc/sphinx-guides/source/contributor/intro.md b/doc/sphinx-guides/source/contributor/intro.md deleted file mode 100644 index 47111d0a663..00000000000 --- a/doc/sphinx-guides/source/contributor/intro.md +++ /dev/null @@ -1,63 +0,0 @@ -# Introduction - -Please note: this is a copy/paste from but we intend to split the content on this intro page into sub-pages. - -Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. We aren't just looking for developers. There are many ways to contribute to Dataverse. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! - -If you have questions, please reach out using one or more of the channels described below. - -```{contents} Contents: -:local: -:depth: 3 -``` - -## Ideas/Feature Requests - -We would love to hear your ideas!💡 - -1. Please check if your idea or feature request is already captured in our [issue tracker][] or [roadmap][]. -1. Bring your idea to the community by posting on our [Google Group][] or [chat.dataverse.org][]. -1. To discuss privately, email us at . - -[issue tracker]: https://github.com/IQSS/dataverse/issues -[roadmap]: https://www.iq.harvard.edu/roadmap-dataverse-project -[chat.dataverse.org]: http://chat.dataverse.org -[Google Group]: https://groups.google.com/group/dataverse-community - -## Bug Reports - -Before submitting an issue, please search for existing issues in our [issue tracker][]. If there is an existing open issue that matches the issue you want to report, please add a comment to it or give it a 👍. - -If there is no pre-existing issue or it has been closed, please open a new issue (unless it is a security issue which should be reported privately to as discussed under {ref}`reporting-security-issues` in the Installation Guide). - -If you do not receive a reply to your new issue or comment in a timely manner, please ping us at [chat.dataverse.org][]. - -## Documentation - -The source for the documentation is in the GitHub repo under the "[doc][]" folder. If you find a typo or inaccuracy or something to clarify, please send us a pull request! For more on the tools used to write docs, please see the {doc}`/developers/documentation` section of the Developer Guide. - -[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source - -## Code/Pull Requests - -We love code contributions. Developers are not limited to the main Dataverse code in this git repo. You can help with API client libraries in your favorite language that are mentioned in the {doc}`/api/index` or create a new library. You can help work on configuration management code that's mentioned in the {doc}`/installation/index` . The Installation Guide also covers a relatively new concept called "external tools" that allows developers to create their own tools that are available from within an installation of Dataverse. - -If you are interested in working on the main Dataverse code, great! Before you start coding, please reach out to us either on the [dataverse-community Google Group][], the [dataverse-dev Google Group][], [chat.dataverse.org][], or via to make sure the effort is well coordinated and we avoid merge conflicts. We maintain a list of [community contributors][] and [dev efforts][] the community is working on so please let us know if you'd like to be added or removed from either list. - -Please read {doc}`/developers/version-control` in the Developer Guide to understand how we use the "git flow" model of development and how we will encourage you to create a GitHub issue (if it doesn't exist already) to associate with your pull request. That page also includes tips on making a pull request. - -After making your pull request, your goal should be to help it advance through our kanban board at . If no one has moved your pull request to the code review column in a timely manner, please reach out. Note that once a pull request is created for an issue, we'll remove the issue from the board so that we only track one card (the pull request). - -Thanks for your contribution! - -## Usability Testing - -Please email us at or fill in our [feedback form][] if you are interested in participating in usability testing. - -[feedback form]: https://goo.gl/forms/p7uu3GfiWYSlJrsi1 - -[dataverse-community Google Group]: https://groups.google.com/group/dataverse-community -[Community Call]: https://dataverse.org/community-calls -[dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev -[community contributors]: https://docs.google.com/spreadsheets/d/1o9DD-MQ0WkrYaEFTD5rF_NtyL8aUISgURsAXSL7Budk/edit?usp=sharing -[dev efforts]: https://github.com/orgs/IQSS/projects/34/views/6 diff --git a/doc/sphinx-guides/source/developers/documentation.rst b/doc/sphinx-guides/source/developers/documentation.rst index a4b8c027445..477776d7ee2 100755 --- a/doc/sphinx-guides/source/developers/documentation.rst +++ b/doc/sphinx-guides/source/developers/documentation.rst @@ -114,6 +114,13 @@ Every non-index page should use the following code to display a table of content This code should be placed below any introductory text/images and directly above the first subheading, much like a Wikipedia page. +If you are using Markdown, you should use this form instead: :: + + ```{contents} Contents: + :local: + :depth: 3 + ``` + Images ------ @@ -125,6 +132,8 @@ While PNGs in the git repo can be linked directly via URL, Sphinx-generated imag provide higher visual quality. Especially in terms of quality of content, generated images can be extendend and improved by a textbased and reviewable commit, without needing raw data or source files and no diff around. +TODO: The above covers "how" but what about "when"? + Cross References ---------------- From f76c71066f3cbb92314f13f8275b041943abae36 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 7 Jun 2024 15:21:05 -0400 Subject: [PATCH 09/31] replace content of CONTRIBUTING.md with link to new guide #10531 --- CONTRIBUTING.md | 66 +++---------------------------------------------- 1 file changed, 3 insertions(+), 63 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1430ba951a6..4fa6e955b70 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,67 +1,7 @@ # Contributing to Dataverse -Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. You don't need permission to participate. Just jump in. If you have questions, please reach out using one or more of the channels described below. +Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. -We aren't just looking for developers. There are many ways to contribute to Dataverse. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! +Please see our [Contributor Guide][] for how you can help! -## Ideas/Feature Requests - -Your idea or feature request might already be captured in the Dataverse [issue tracker] on GitHub but if not, the best way to bring it to the community's attention is by posting on the [dataverse-community Google Group][] or bringing it up on a [Community Call][]. You're also welcome to make some noise in [chat.dataverse.org][] or cram your idea into 280 characters and mention [@dataverseorg][] on Twitter. To discuss your idea privately, please email it to support@dataverse.org - -There's a chance your idea is already on our roadmap, which is available at https://www.iq.harvard.edu/roadmap-dataverse-project - -[chat.dataverse.org]: http://chat.dataverse.org -[issue tracker]: https://github.com/IQSS/dataverse/issues -[@dataverseorg]: https://twitter.com/dataverseorg - -## Usability testing - -Please email us at support@dataverse.org if you are interested in participating in usability testing. - -## Bug Reports/Issues - -An issue is a bug (a feature is no longer behaving the way it should) or a feature (something new to Dataverse that helps users complete tasks). You can browse the Dataverse [issue tracker] on GitHub by open or closed issues or by milestones. - -Before submitting an issue, please search the existing issues by using the search bar at the top of the page. If there is an existing open issue that matches the issue you want to report, please add a comment to it. - -If there is no pre-existing issue or it has been closed, please click on the "New Issue" button, log in, and write in what the issue is (unless it is a security issue which should be reported privately to security@dataverse.org). - -If you do not receive a reply to your new issue or comment in a timely manner, please email support@dataverse.org with a link to the issue. - -### Writing an Issue - -For the subject of an issue, please start it by writing the feature or functionality it relates to, i.e. "Create Account:..." or "Dataset Page:...". In the body of the issue, please outline the issue you are reporting with as much detail as possible. In order for the Dataverse development team to best respond to the issue, we need as much information about the issue as you can provide. Include steps to reproduce bugs. Indicate which version you're using, which is shown at the bottom of the page. We love screenshots! - -### Issue Attachments - -You can attach certain files (images, screenshots, logs, etc.) by dragging and dropping, selecting them, or pasting from the clipboard. Files must be one of GitHub's [supported attachment formats] such as png, gif, jpg, txt, pdf, zip, etc. (Pro tip: A file ending in .log can be renamed to .txt so you can upload it.) If there's no easy way to attach your file, please include a URL that points to the file in question. - -[supported attachment formats]: https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/ - -## Documentation - -The source for the documentation at http://guides.dataverse.org/en/latest/ is in the GitHub repo under the "[doc][]" folder. If you find a typo or inaccuracy or something to clarify, please send us a pull request! For more on the tools used to write docs, please see the [documentation][] section of the Developer Guide. - -[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source -[documentation]: http://guides.dataverse.org/en/latest/developers/documentation.html - -## Code/Pull Requests - -We love code contributions. Developers are not limited to the main Dataverse code in this git repo. You can help with API client libraries in your favorite language that are mentioned in the [API Guide][] or create a new library. You can help work on configuration management code that's mentioned in the [Installation Guide][]. The Installation Guide also covers a relatively new concept called "external tools" that allows developers to create their own tools that are available from within an installation of Dataverse. - -[API Guide]: http://guides.dataverse.org/en/latest/api -[Installation Guide]: http://guides.dataverse.org/en/latest/installation - -If you are interested in working on the main Dataverse code, great! Before you start coding, please reach out to us either on the [dataverse-community Google Group][], the [dataverse-dev Google Group][], [chat.dataverse.org][], or via support@dataverse.org to make sure the effort is well coordinated and we avoid merge conflicts. We maintain a list of [community contributors][] and [dev efforts][] the community is working on so please let us know if you'd like to be added or removed from either list. - -Please read http://guides.dataverse.org/en/latest/developers/version-control.html to understand how we use the "git flow" model of development and how we will encourage you to create a GitHub issue (if it doesn't exist already) to associate with your pull request. That page also includes tips on making a pull request. - -After making your pull request, your goal should be to help it advance through our kanban board at https://github.com/orgs/IQSS/projects/34 . If no one has moved your pull request to the code review column in a timely manner, please reach out. Note that once a pull request is created for an issue, we'll remove the issue from the board so that we only track one card (the pull request). - -Thanks for your contribution! - -[dataverse-community Google Group]: https://groups.google.com/group/dataverse-community -[Community Call]: https://dataverse.org/community-calls -[dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev -[community contributors]: https://docs.google.com/spreadsheets/d/1o9DD-MQ0WkrYaEFTD5rF_NtyL8aUISgURsAXSL7Budk/edit?usp=sharing -[dev efforts]: https://github.com/orgs/IQSS/projects/34/views/6 +[Contributor Guide]: https://guides.dataverse.org/en/latest/contributor/index.html From 36b4ed99636f55f794c332f6fff8d829ea691a9a Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Mon, 10 Jun 2024 23:02:47 -0400 Subject: [PATCH 10/31] expand "contributing code", other tweaks #10531 --- doc/sphinx-guides/source/contributor/code.md | 54 +++++++++++++++---- doc/sphinx-guides/source/developers/intro.rst | 26 +++++---- .../source/developers/version-control.rst | 2 + 3 files changed, 61 insertions(+), 21 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/code.md b/doc/sphinx-guides/source/contributor/code.md index 941ce1fd6bc..9bc2b5a2027 100644 --- a/doc/sphinx-guides/source/contributor/code.md +++ b/doc/sphinx-guides/source/contributor/code.md @@ -1,26 +1,58 @@ # Contributing Code +We love code contributions! There are lots of ways you can help. + ```{contents} Contents: :local: :depth: 3 ``` -## Welcome! +## Finding an Issue to Work On + +New contributors often wonder what issues they should work on first. + +### Many Codebases, Many Languages + +The primary codebase and issue tracker for Dataverse is . It's mostly backend code written in Java. However, there are many other codebases you can work on in a variety of languages. Here are a few that are especially active: + +- (Java) +- (React) +- (TypeScript) +- (Javascript) +- (Python) +- (Ansible) +- (Javascript) + +If nothing above sparks joy, you can find more projects to work on under {doc}`/api/client-libraries`, {doc}`/api/external-tools`, {ref}`related-projects`, and {doc}`/api/apps`. + +### Picking a Good First Issue + +Once you've decided which codebase suits you, you should try to identify an issue to work on. Some codebases use a label like "good first issue" to suggest issues for newcomers. -We love code contributions. Developers are not limited to the main Dataverse code in this git repo. You can help with API {doc}`/api/client-libraries` in your favorite language or create a new library. You can help work on configuration management code that's mentioned in the {doc}`/installation/index`. See {doc}`/api/external-tools` for how you can create your own tool that is available from within an installation of Dataverse. +For the main codebase, please see {ref}`finding-github-issues-to-work-on` which includes information on labels like "good first issue". -If you are interested in working on the main Dataverse code, great! Before you start coding, please reach out to us either on the [dataverse-community Google Group][], the [dataverse-dev Google Group][], [chat.dataverse.org][], or via to make sure the effort is well coordinated and we avoid merge conflicts. We maintain a list of [community contributors][] and [dev efforts][] the community is working on so please let us know if you'd like to be added or removed from either list. +Other codebases may use different labels. Check the README or other documentation for that codebase. + +If there is a linked pull request that is trying to close the issue, you should probably find another issue. + +If you are having trouble finding an issue or have any questions at all, please do not hesitate to reach out. See {ref}`getting-help-developers`. ## Making a Pull Request -Please read {doc}`/developers/version-control` in the Developer Guide to understand how we use the "git flow" model of development and how we will encourage you to create a GitHub issue (if it doesn't exist already) to associate with your pull request. That page also includes tips on making a pull request. +For the main codebase, please see {ref}`how-to-make-a-pull-request`. + +For other codebases, consult the README. + +## Reviewing Code + +Reviewing code is a great way to learn about a codebase. For any codebase you can browse open pull requests, of course, but for the primary codebases, you can take a look at the "Ready for Review" and "In Review" columns at https://github.com/orgs/IQSS/projects/34 + +You are welcome to review code informally or to leave an actual review. We're interested in what you think. + +## Reproducing Bugs -After making your pull request, your goal should be to help it advance through our kanban board at . If no one has moved your pull request to the code review column in a timely manner, please reach out. Note that once a pull request is created for an issue, we'll remove the issue from the board so that we only track one card (the pull request). +At times, bugs are reported that we haven't had time to confirm. You can help out by reproducing bugs and commenting on the issue the results you find. -Thanks for your contribution! +## Getting Help -[dataverse-community Google Group]: https://groups.google.com/group/dataverse-community -[dataverse-dev Google Group]: https://groups.google.com/group/dataverse-dev -[chat.dataverse.org]: http://chat.dataverse.org -[community contributors]: https://docs.google.com/spreadsheets/d/1o9DD-MQ0WkrYaEFTD5rF_NtyL8aUISgURsAXSL7Budk/edit?usp=sharing -[dev efforts]: https://github.com/orgs/IQSS/projects/34/views/6 +If you have any questions at all, please do not hesitate to reach out. See {ref}`getting-help-developers`. diff --git a/doc/sphinx-guides/source/developers/intro.rst b/doc/sphinx-guides/source/developers/intro.rst index 350968012d8..0e74dc1c36f 100755 --- a/doc/sphinx-guides/source/developers/intro.rst +++ b/doc/sphinx-guides/source/developers/intro.rst @@ -2,7 +2,7 @@ Introduction ============ -Welcome! `The Dataverse Project `_ is an `open source `_ project that loves `contributors `_! +Welcome! `The Dataverse Project `_ is an `open source `_ project that loves contributors! .. contents:: |toctitle| :local: @@ -10,7 +10,7 @@ Welcome! `The Dataverse Project `_ is an `open source `_ mailing list, `community calls `_, or support@dataverse.org. +If you have any questions at all, please reach out to other developers via https://chat.dataverse.org, the `dataverse-dev `_ mailing list, the `dataverse-community `_ mailing list, or `community calls `_. .. _core-technologies: Core Technologies ----------------- -The Dataverse Software is a `Jakarta EE `_ application that is compiled into a WAR file and deployed to an application server (app server) which is configured to work with a relational database (PostgreSQL) and a search engine (Solr). +Dataverse is a `Jakarta EE `_ application that is compiled into a WAR file and deployed to an application server (app server) which is configured to work with a relational database (PostgreSQL) and a search engine (Solr). -We make use of a variety of Jakarta EE technologies such as JPA, JAX-RS, JMS, and JSF. The front end is built using PrimeFaces and Bootstrap. +We make use of a variety of Jakarta EE technologies such as JPA, JAX-RS, JMS, and JSF. In addition, we use parts of Eclipse MicroProfile such as `MicroProfile Config `_. -In addition, we start to adopt parts of Eclipse MicroProfile, namely `MicroProfile Config `_. +The frontend is built using PrimeFaces and Bootstrap. A new frontend is being built using React at https://github.com/IQSS/dataverse-frontend Roadmap ------- -For the Dataverse Software development roadmap, please see https://www.iq.harvard.edu/roadmap-dataverse-project +For the roadmap, please see https://www.iq.harvard.edu/roadmap-dataverse-project .. _kanban-board: @@ -47,20 +47,26 @@ You can get a sense of what's currently in flight (in dev, in QA, etc.) by looki Issue Tracker ------------- -We use GitHub Issues as our issue tracker: https://github.com/IQSS/dataverse/issues +The main issue tracker is https://github.com/IQSS/dataverse/issues but note that individual projects have their own issue trackers. Related Guides -------------- -If you are a developer who wants to make use of the Dataverse Software APIs, please see the :doc:`/api/index`. If you have front-end UI questions, please see the :doc:`/style/index`. +If you are wondering about how to contribute generally, please see the :doc:`/contributor/index`. -If you are a sysadmin who likes to code, you may be interested in hacking on installation scripts mentioned in the :doc:`/installation/index`. +If you are a developer who wants to make use of the Dataverse APIs, please see the :doc:`/api/index`. + +If you have frontend UI questions, please see the :doc:`/style/index`. For the new frontend, see https://github.com/IQSS/dataverse-frontend If you are a Docker enthusiasts, please check out the :doc:`/container/index`. +.. _related-projects: + Related Projects ---------------- +Note: this list is somewhat old. Please see also the :doc:`/contributor/code` section of the Contributor Guide. + As a developer, you also may be interested in these projects related to Dataverse: - External Tools - add additional features to the Dataverse Software without modifying the core: :doc:`/api/external-tools` diff --git a/doc/sphinx-guides/source/developers/version-control.rst b/doc/sphinx-guides/source/developers/version-control.rst index 07922b56b86..8648c8ce2a0 100644 --- a/doc/sphinx-guides/source/developers/version-control.rst +++ b/doc/sphinx-guides/source/developers/version-control.rst @@ -69,6 +69,8 @@ Find or Create a GitHub Issue An issue represents a bug (unexpected behavior) or a new feature in Dataverse. We'll use the issue number in the branch we create for our pull request. +.. _finding-github-issues-to-work-on: + Finding GitHub Issues to Work On ******************************** From a5c42c1825c5474cdd6f67fecf0a9b6b6cfa6ae5 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 11 Jun 2024 07:40:59 -0400 Subject: [PATCH 11/31] add translations to contrib guide #10531 --- doc/sphinx-guides/source/contributor/index.md | 4 ++++ doc/sphinx-guides/source/installation/config.rst | 2 ++ 2 files changed, 6 insertions(+) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 7a965b994e1..819a1d8a438 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -42,6 +42,10 @@ Please observe the following when writing documentation: [doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source +## Translations + +If you speak multiple languages, you are very welcome to help us translate Dataverse! Please see {ref}`help-translate` for details. + ## Code and Pull Requests Dataverse is open source and we love code contributions. Developers are not limited to the main Dataverse code in this git repo. We have projects in C, C++, Go, Java, Javascript, Julia, PHP, Python, R, Ruby, TypeScript and more. To get started, please see the following pages: diff --git a/doc/sphinx-guides/source/installation/config.rst b/doc/sphinx-guides/source/installation/config.rst index 8fb9460892b..6d0da625a5a 100644 --- a/doc/sphinx-guides/source/installation/config.rst +++ b/doc/sphinx-guides/source/installation/config.rst @@ -1741,6 +1741,8 @@ Now that you have a "languages.zip" file, you can load it into your Dataverse in Click on the languages using the drop down in the header to try them out. +.. _help-translate: + How to Help Translate the Dataverse Software Into Your Language +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From be05cadfe307b164aee085e846662210b003fe08 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 11 Jun 2024 08:22:53 -0400 Subject: [PATCH 12/31] add support, talks, working groups #10531 --- doc/sphinx-guides/source/contributor/index.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 819a1d8a438..78491578c5f 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -60,3 +60,15 @@ code.md Please email us at or fill in our [feedback form][] if you are interested in participating in usability testing. [feedback form]: https://goo.gl/forms/p7uu3GfiWYSlJrsi1 + +## Answering Questions + +People come along with questions on the [mailing list](https://groups.google.com/g/dataverse-community) and in [chat](https://chat.dataverse.org) all the time. You are very welcome to help out by answering these questions to the best of your ability. + +## Giving Talks + +If you give a recorded talk about Dataverse, we are happy to add it to [DataverseTV](https://dataverse.org/dataversetv) + +## Working Groups + +Most working groups are wide open to participation. For the current list of groups, please see . From 13c814dd706df7f3e25ca73350c581187f052e6e Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Tue, 11 Jun 2024 12:37:44 -0400 Subject: [PATCH 13/31] add issue triage, other tweaks #10531 --- doc/sphinx-guides/source/contributor/index.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 78491578c5f..5168b259d25 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -63,11 +63,21 @@ Please email us at or fill in our [feedback form][] if y ## Answering Questions -People come along with questions on the [mailing list](https://groups.google.com/g/dataverse-community) and in [chat](https://chat.dataverse.org) all the time. You are very welcome to help out by answering these questions to the best of your ability. +People come along with questions on the [mailing list](https://groups.google.com/g/dataverse-community) and in [chat][] all the time. You are very welcome to help out by answering these questions to the best of your ability. + +[chat]: https://chat.dataverse.org + +## Issue Triage + +New issues are coming in all the time, especially for the main issue tracker at . + +You can help by leaving comments that mention related issues or that answer questions. ## Giving Talks -If you give a recorded talk about Dataverse, we are happy to add it to [DataverseTV](https://dataverse.org/dataversetv) +If you give a recorded talk about Dataverse, we are happy to add it to [DataverseTV](https://dataverse.org/dataversetv). You can just leave a comment on the [spreadsheet](https://docs.google.com/spreadsheets/d/1uVk_57Ek_A49sLZ5OKdI6QASKloWNzykni3kcYNzpxA/edit#gid=0) or make some noise in [chat][]. + +For non-recorded talks, we are happy to upload your slides to . Please email . ## Working Groups From 22d9a2008281a15f49b00cc1ebc60ac7b8bc823e Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 12 Jun 2024 17:09:08 -0400 Subject: [PATCH 14/31] add sample data, artwork, other --- doc/sphinx-guides/source/contributor/index.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 5168b259d25..83a2f59003c 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -67,6 +67,10 @@ People come along with questions on the [mailing list](https://groups.google.com [chat]: https://chat.dataverse.org +## Sample Data + +Consider contributing to , a git repo of realistic-looking data that is used for testing. + ## Issue Triage New issues are coming in all the time, especially for the main issue tracker at . @@ -82,3 +86,17 @@ For non-recorded talks, we are happy to upload your slides to . + +## Artwork + +Surely we can put artistic talent to use. See for a story about drawing animals chatting about Dataverse. + +As [annnounced](https://groups.google.com/g/dataverse-community/c/pM39_9O5Rug/m/CK-gJqZFBgAJ), we have a [sticker template](https://dataverse.org/sites/projects.iq.harvard.edu/files/dataverseorg/files/dataverse_community_stickers_template.zip) you can use. + +See [Illustrations from The Turing Way](https://zenodo.org/doi/10.5281/zenodo.3332807) for how that community has created artwork. Perhaps we can do the same. + +## Other Contributions + +We consulted but no list is comprehensive. + +What else should we list here? How to YOU want to contribute to Dataverse? 🎉 From 0b7cb9f354574064441b087732c12e5cfe0a9495 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Wed, 12 Jun 2024 17:25:55 -0400 Subject: [PATCH 15/31] move documentation into own sub page of contrib guide #10531 --- .../source/contributor/documentation.md | 28 +++++++++++++++++++ doc/sphinx-guides/source/contributor/index.md | 15 ++++------ .../source/developers/documentation.rst | 2 ++ 3 files changed, 35 insertions(+), 10 deletions(-) create mode 100644 doc/sphinx-guides/source/contributor/documentation.md diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md new file mode 100644 index 00000000000..404f26982d1 --- /dev/null +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -0,0 +1,28 @@ +# Writing Documentation + +Thank you for your interest in contributing documentation to Dataverse! Good documentation is absolutely critical to the success of a software project. + +```{contents} Contents: +:local: +:depth: 3 +``` + +## Where the Source Files Live + +The source for the documentation is in the main "dataverse" GitHub repo under the "[doc][]" folder. + +[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source + +## Tools + +The {doc}`/developers/documentation` section of the Developer Guide has, for a long time, been the primary place to find information about the tools we use to write docs. We may move some of that content here, but until then, please visit that page. + +## Writing Style Guidelines + +Please observe the following when writing documentation: + +- Use American English spelling. +- Use examples when possible. +- Break up longer paragraphs. +- Use "double quotes" instead of 'single quotes'. +- Favor "and" (data and code) over slashes (data/code). diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 83a2f59003c..c13f6765a38 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -30,17 +30,12 @@ If you do not receive a reply to your new issue or comment in a timely manner, p ## Documentation -The source for the documentation is in the GitHub repo under the "[doc][]" folder. If you find a typo or inaccuracy or something to clarify, please send us a pull request! For more on the tools used to write docs, please see the {doc}`/developers/documentation` section of the Developer Guide. +Documentation is such a large topic (and important!) that we have a dedicate section on it: -Please observe the following when writing documentation: - -- Use American English spelling. -- Use examples when possible. -- Break up longer paragraphs. -- Use "double quotes" instead of 'single quotes'. -- Favor "and" (data and code) over slashes (data/code). - -[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source +```{toctree} +:maxdepth: 1 +documentation.md +``` ## Translations diff --git a/doc/sphinx-guides/source/developers/documentation.rst b/doc/sphinx-guides/source/developers/documentation.rst index 477776d7ee2..f0ee2dcabf1 100755 --- a/doc/sphinx-guides/source/developers/documentation.rst +++ b/doc/sphinx-guides/source/developers/documentation.rst @@ -2,6 +2,8 @@ Writing Documentation ===================== +Note: in the Contributor Guide, there is a section called :doc:`/contributor/documentation`. + .. contents:: |toctitle| :local: From fb461c77d9a0929f992cede57e40bc1ebe683465 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 09:32:08 -0400 Subject: [PATCH 16/31] shorted to just "code" #10531 --- doc/sphinx-guides/source/contributor/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index c13f6765a38..e603d6151b7 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -41,7 +41,7 @@ documentation.md If you speak multiple languages, you are very welcome to help us translate Dataverse! Please see {ref}`help-translate` for details. -## Code and Pull Requests +## Code Dataverse is open source and we love code contributions. Developers are not limited to the main Dataverse code in this git repo. We have projects in C, C++, Go, Java, Javascript, Julia, PHP, Python, R, Ruby, TypeScript and more. To get started, please see the following pages: From b0808424de5ed85fe5e6ecd7d2e788ae027c0ab0 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 09:46:24 -0400 Subject: [PATCH 17/31] link to new "writing docs" under contrib guide #10531 --- doc/sphinx-guides/source/admin/integrations.rst | 2 +- doc/sphinx-guides/source/api/faq.rst | 4 ++-- doc/sphinx-guides/source/installation/config.rst | 2 +- doc/sphinx-guides/source/installation/intro.rst | 2 +- doc/sphinx-guides/source/qa/testing-infrastructure.md | 2 +- doc/sphinx-guides/source/versions.rst | 2 +- 6 files changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/sphinx-guides/source/admin/integrations.rst b/doc/sphinx-guides/source/admin/integrations.rst index e48d7a60798..7a9b9c92f77 100644 --- a/doc/sphinx-guides/source/admin/integrations.rst +++ b/doc/sphinx-guides/source/admin/integrations.rst @@ -290,4 +290,4 @@ If you have an idea for an integration, please ask on the `dataverse-community < Many integrations take the form of "external tools". See the :doc:`external-tools` section for details. External tool makers should check out the :doc:`/api/external-tools` section of the API Guide. -Please help us keep this page up to date making a pull request! To get started, see the :doc:`/developers/documentation` section of the Developer Guide. +Please help us keep this page up to date making a pull request! To get started, see the :doc:`/contributor/documentation` section of the Contributor Guide. diff --git a/doc/sphinx-guides/source/api/faq.rst b/doc/sphinx-guides/source/api/faq.rst index 439783779c3..c335b30a0e0 100644 --- a/doc/sphinx-guides/source/api/faq.rst +++ b/doc/sphinx-guides/source/api/faq.rst @@ -82,12 +82,12 @@ The following tasks cannot currently be automated via API because no API exists If you would like APIs for any of the features above, please open a GitHub issue at https://github.com/IQSS/dataverse/issues -You are also welcome to open an issue to add to the list above. Or you are welcome to make a pull request. Please see the :doc:`/developers/documentation` section of the Developer Guide for instructions. +You are also welcome to open an issue to add to the list above. Or you are welcome to make a pull request. Please see the :doc:`/contributor/documentation` section of the Contributor Guide for instructions. Why Are the Return Values (HTTP Status Codes) Not Documented? ------------------------------------------------------------- -They should be. Please consider making a pull request to help. The :doc:`/developers/documentation` section of the Developer Guide should help you get started. :ref:`create-dataverse-api` has an example you can follow or you can come up with a better way. +They should be. Please consider making a pull request to help. The :doc:`/contributor/documentation` section of the Contriburor Guide should help you get started. :ref:`create-dataverse-api` has an example you can follow or you can come up with a better way. Also, please note that we are starting to experiment with putting response codes in our OpenAPI document. See :ref:`openapi`. diff --git a/doc/sphinx-guides/source/installation/config.rst b/doc/sphinx-guides/source/installation/config.rst index 6d0da625a5a..e2f7048b6ff 100644 --- a/doc/sphinx-guides/source/installation/config.rst +++ b/doc/sphinx-guides/source/installation/config.rst @@ -3723,7 +3723,7 @@ Note: by default, the URL is composed from the settings ``:GuidesBaseUrl`` and ` :GuidesBaseUrl ++++++++++++++ -Set ``:GuidesBaseUrl`` to override the default value "https://guides.dataverse.org". If you are interested in writing your own version of the guides, you may find the :doc:`/developers/documentation` section of the Developer Guide helpful. +Set ``:GuidesBaseUrl`` to override the default value "https://guides.dataverse.org". If you are interested in writing your own version of the guides, you may find the :doc:`/contributor/documentation` section of the Contributor Guide helpful. ``curl -X PUT -d http://dataverse.example.edu http://localhost:8080/api/admin/settings/:GuidesBaseUrl`` diff --git a/doc/sphinx-guides/source/installation/intro.rst b/doc/sphinx-guides/source/installation/intro.rst index 6d77a1209b2..1c239863e98 100644 --- a/doc/sphinx-guides/source/installation/intro.rst +++ b/doc/sphinx-guides/source/installation/intro.rst @@ -53,6 +53,6 @@ If you've encountered a problem installing Dataverse and are ready to ask for he Improving this Guide -------------------- -If you spot a typo in this guide or would like to suggest an improvement, please find the appropriate file in https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/installation and send a pull request as explained in the :doc:`/developers/documentation` section of the Developer Guide. You are also welcome to simply open an issue at https://github.com/IQSS/dataverse/issues to describe the problem with this guide. +If you spot a typo in this guide or would like to suggest an improvement, please find the appropriate file in https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/installation and send a pull request as explained in the :doc:`/contributor/documentation` section of the Contributor Guide. You are also welcome to simply open an issue at https://github.com/IQSS/dataverse/issues to describe the problem with this guide. Next is the :doc:`prep` section. diff --git a/doc/sphinx-guides/source/qa/testing-infrastructure.md b/doc/sphinx-guides/source/qa/testing-infrastructure.md index 804e4c0afe6..6ec26c6da49 100644 --- a/doc/sphinx-guides/source/qa/testing-infrastructure.md +++ b/doc/sphinx-guides/source/qa/testing-infrastructure.md @@ -39,7 +39,7 @@ To build and test a PR, we use a job called `IQSS_Dataverse_Internal` on . This Read the Docs preview is also mentioned under also {doc}`/developers/documentation`. +Note that changes to guides can also be previewed on Read the Docs. In the pull request, look for a link like . This Read the Docs preview is also mentioned under also {doc}`/contributor/documentation`. ## Other Servers diff --git a/doc/sphinx-guides/source/versions.rst b/doc/sphinx-guides/source/versions.rst index d76f9a889cb..850702d823e 100755 --- a/doc/sphinx-guides/source/versions.rst +++ b/doc/sphinx-guides/source/versions.rst @@ -6,7 +6,7 @@ Dataverse Software Documentation Versions This list provides a way to refer to the documentation for previous and future versions of the Dataverse Software. In order to learn more about the updates delivered from one version to another, visit the `Releases `__ page in our GitHub repo. -- pre-release `HTML (not final!) `__ and `PDF (experimental!) `__ built from the :doc:`develop ` branch :doc:`(how to contribute!) ` +- pre-release `HTML (not final!) `__ and `PDF (experimental!) `__ built from the :doc:`develop ` branch :doc:`(how to contribute!) ` - 6.2 - `6.1 `__ - `6.0 `__ From 55ce252b047a793c900a4844e7922eca83daf83e Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 10:25:32 -0400 Subject: [PATCH 18/31] move toc #10531 --- .../source/contributor/documentation.md | 16 ++++++++++++++++ .../source/developers/documentation.rst | 17 ----------------- 2 files changed, 16 insertions(+), 17 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 404f26982d1..a5f5bc2d93e 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -26,3 +26,19 @@ Please observe the following when writing documentation: - Break up longer paragraphs. - Use "double quotes" instead of 'single quotes'. - Favor "and" (data and code) over slashes (data/code). + +## Table of Contents + +Every non-index page should use the following code to display a table of contents of internal sub-headings. This code should be placed below any introductory text and directly above the first subheading, much like a Wikipedia page. + +If the page is written in reStructuredText (.rst), use this form: + + .. contents:: |toctitle| + :local: + +If the page is written in Markdown (.md), use this form: + + ```{contents} Contents: + :local: + :depth: 3 + ``` diff --git a/doc/sphinx-guides/source/developers/documentation.rst b/doc/sphinx-guides/source/developers/documentation.rst index f0ee2dcabf1..c90b46a4ce9 100755 --- a/doc/sphinx-guides/source/developers/documentation.rst +++ b/doc/sphinx-guides/source/developers/documentation.rst @@ -106,23 +106,6 @@ You can click on the files in the ``html`` folder to preview the changes. Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See :ref:`how-to-make-a-pull-request`. -Table of Contents ------------------ - -Every non-index page should use the following code to display a table of contents of internal sub-headings: :: - - .. contents:: |toctitle| - :local: - -This code should be placed below any introductory text/images and directly above the first subheading, much like a Wikipedia page. - -If you are using Markdown, you should use this form instead: :: - - ```{contents} Contents: - :local: - :depth: 3 - ``` - Images ------ From c243a9ca325fa35ccd4147d8df6c931efa7c58fd Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 10:30:33 -0400 Subject: [PATCH 19/31] move images and cross references #10531 --- .../source/contributor/documentation.md | 12 ++++++++++++ .../source/developers/documentation.rst | 18 ------------------ 2 files changed, 12 insertions(+), 18 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index a5f5bc2d93e..a659e4467dc 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -42,3 +42,15 @@ If the page is written in Markdown (.md), use this form: :local: :depth: 3 ``` + +## Images + +A good documentation is just like a website enhanced and upgraded by adding high quality and self-explanatory images. Often images depict a lot of written text in a simple manner. Within our Sphinx docs, you can add them in two ways: a) add a PNG image directly and include or b) use inline description languages like GraphViz (current only option). + +While PNGs in the git repo can be linked directly via URL, Sphinx-generated images do not need a manual step and might provide higher visual quality. Especially in terms of quality of content, generated images can be extendend and improved by a textbased and reviewable commit, without needing raw data or source files and no diff around. + +TODO: The above covers "how" but what about "when"? + +## Cross References + +When adding ReStructured Text (.rst) [cross references](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role), use the hyphen character (`-`) as the word separator for the cross reference label. For example, `my-reference-label` would be the preferred label for a cross reference as opposed to, for example, `my_reference_label`. diff --git a/doc/sphinx-guides/source/developers/documentation.rst b/doc/sphinx-guides/source/developers/documentation.rst index c90b46a4ce9..8ff8d1477d8 100755 --- a/doc/sphinx-guides/source/developers/documentation.rst +++ b/doc/sphinx-guides/source/developers/documentation.rst @@ -106,24 +106,6 @@ You can click on the files in the ``html`` folder to preview the changes. Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See :ref:`how-to-make-a-pull-request`. -Images ------- - -A good documentation is just like a website enhanced and upgraded by adding high quality and self-explanatory images. -Often images depict a lot of written text in a simple manner. Within our Sphinx docs, you can add them in two ways: a) add a -PNG image directly and include or b) use inline description languages like GraphViz (current only option). - -While PNGs in the git repo can be linked directly via URL, Sphinx-generated images do not need a manual step and might -provide higher visual quality. Especially in terms of quality of content, generated images can be extendend and improved -by a textbased and reviewable commit, without needing raw data or source files and no diff around. - -TODO: The above covers "how" but what about "when"? - -Cross References ----------------- - -**NOTE:** When adding ReStructured Text (RST) `cross references `_, use the hyphen character (``-``) as the word separator for the cross reference label. For example, ``my-reference-label`` would be the preferred label for a cross reference as opposed to, for example, ``my_reference_label``. - Versions -------- From a970df20a5a8146d7ce5bb2b5679f9787d16343b Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 10:51:53 -0400 Subject: [PATCH 20/31] move writing docs page from dev to contrib guide #10531 --- .../source/contributor/documentation.md | 115 ++++++++++++++- .../source/developers/documentation.rst | 137 ------------------ doc/sphinx-guides/source/developers/index.rst | 1 - 3 files changed, 110 insertions(+), 143 deletions(-) delete mode 100755 doc/sphinx-guides/source/developers/documentation.rst diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index a659e4467dc..e0b83560870 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -7,15 +7,120 @@ Thank you for your interest in contributing documentation to Dataverse! Good doc :depth: 3 ``` -## Where the Source Files Live +## Quick Fix -The source for the documentation is in the main "dataverse" GitHub repo under the "[doc][]" folder. +If you find a typo or a small error in the documentation you can fix it using GitHub's online web editor. Generally speaking, we will be following -[doc]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source +- Navigate to where you will see folders for each of the guides: [admin][], [api][], [container][], etc. +- Find the file you want to edit under one of the folders above. +- Click the pencil icon in the upper-right corner. If this is your first contribution to the Dataverse Project, the hover text over the pencil icon will say "Fork this project and edit this file". +- Make changes to the file and preview them. +- In the **Commit changes** box, enter a description of the changes you have made and click **Propose file change**. +- Under the **Write** tab, delete the long welcome message and write a few words about what you fixed. +- Click **Create Pull Request**. -## Tools +That's it! Thank you for your contribution! Your pull request will be added manually to the main Dataverse Project board at and will go through code review and QA before it is merged into the "develop" branch. Along the way, developers might suggest changes or make them on your behalf. Once your pull request has been merged you will be listed as a contributor at ! 🎉 -The {doc}`/developers/documentation` section of the Developer Guide has, for a long time, been the primary place to find information about the tools we use to write docs. We may move some of that content here, but until then, please visit that page. +Please see for an example of a quick fix that was merged (the "Files changed" tab shows how a typo was fixed). + +Preview your documentation changes which will be built automatically as part of your pull request in Github. It will show up as a check entitled: `docs/readthedocs.org:dataverse-guide — Read the Docs build succeeded!`. For example, this PR built to . + +If you would like to read more about the Dataverse Project's use of GitHub, please see the {doc}`/developers/version-control` section of the Developer Guide. For bug fixes and features we request that you create an issue before making a pull request but this is not at all necessary for quick fixes to the documentation. + +[admin]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/admin +[api]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/api +[container]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/container + +## Building the Guides with Sphinx + +The Dataverse guides are written using Sphinx (). We recommend installing Sphinx on your computer or using a Sphinx Docker container to build the guides locally so you can get an accurate preview of your changes. + +In case you decide to use a Sphinx Docker container to build the guides, you can skip the next two installation sections, but you will need to have Docker installed. + +### Installing Sphinx + +First, make a fork of https://github.com/IQSS/dataverse and clone your fork locally. Then change to the ``doc/sphinx-guides`` directory. + +``cd doc/sphinx-guides`` + +Create a Python virtual environment, activate it, then install dependencies: + +``python3 -m venv venv`` + +``source venv/bin/activate`` + +``pip install -r requirements.txt`` + +### Installing GraphViz + +In some parts of the documentation, graphs are rendered as images using the Sphinx GraphViz extension. + +Building the guides requires the ``dot`` executable from GraphViz. + +This requires having `GraphViz `_ installed and either having ``dot`` on the path or +`adding options to the make call `_. + +### Editing and Building the Guides + +To edit the existing documentation: + +- Create a branch (see :ref:`how-to-make-a-pull-request`). +- In ``doc/sphinx-guides/source`` you will find the .rst files that correspond to https://guides.dataverse.org. +- Using your preferred text editor, open and edit the necessary files, or create new ones. + +Once you are done, you can preview the changes by building the guides locally. As explained, you can build the guides with Sphinx locally installed, or with a Docker container. + +#### Building the Guides with Sphinx Locally Installed + +Open a terminal, change directories to `doc/sphinx-guides`, activate (or reactivate) your Python virtual environment, and build the guides. + +`cd doc/sphinx-guides` + +`source venv/bin/activate` + +`make clean` + +`make html` + +#### Building the Guides with a Sphinx Docker Container + +If you want to build the guides using a Docker container, execute the following command in the repository root: + +`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make html"` + +#### Previewing the Guides + +After Sphinx is done processing the files you should notice that the `html` folder in `doc/sphinx-guides/build` directory has been updated. You can click on the files in the `html` folder to preview the changes. + +Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See {ref}`how-to-make-a-pull-request`. + +## Versions + +For installations hosting their own copies of the guides, note that as each version of the Dataverse Software is released, there is an updated version of the guides released with it. Google and other search engines index all versions, which may confuse users who discover your guides in the search results as to which version they should be looking at. When learning about your installation from the search results, it is best to be viewing the *latest* version. + +In order to make it clear to the crawlers that we only want the latest version discoverable in their search results, we suggest adding this to your `robots.txt` file: + + User-agent: * + Allow: /en/latest/ + Disallow: /en/ + +## PDF Version of the Guides + +The HTML version of the guides is the official one. Any other formats are maintained on a best effort basis. + +If you would like to build a PDF version of the guides and have Docker installed, please try the command below from the root of the git repo: + +`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"` + +A few notes about the command above: + +- Hopefully the PDF was created at `doc/sphinx-guides/build/latex/Dataverse.pdf`. +- For now, we are using "nonstopmode" but this masks some errors. +- See requirements.txt for a note regarding the version of Sphinx we are using. + +Also, as of this writing we have enabled PDF builds from the "develop" branch. You download the PDF from + +If you would like to help improve the PDF version of the guides, please get in touch! Please see {ref}`getting-help-developers` for ways to contact the developer community. ## Writing Style Guidelines diff --git a/doc/sphinx-guides/source/developers/documentation.rst b/doc/sphinx-guides/source/developers/documentation.rst deleted file mode 100755 index 8ff8d1477d8..00000000000 --- a/doc/sphinx-guides/source/developers/documentation.rst +++ /dev/null @@ -1,137 +0,0 @@ -===================== -Writing Documentation -===================== - -Note: in the Contributor Guide, there is a section called :doc:`/contributor/documentation`. - -.. contents:: |toctitle| - :local: - -Quick Fix ------------ - -If you find a typo or a small error in the documentation you can fix it using GitHub's online web editor. Generally speaking, we will be following https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files#editing-files-in-another-users-repository - -- Navigate to https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source where you will see folders for each of the guides: `admin`_, `api`_, `developers`_, `installation`_, `style`_, `user`_. -- Find the file you want to edit under one of the folders above. -- Click the pencil icon in the upper-right corner. If this is your first contribution to the Dataverse Project, the hover text over the pencil icon will say "Fork this project and edit this file". -- Make changes to the file and preview them. -- In the **Commit changes** box, enter a description of the changes you have made and click **Propose file change**. -- Under the **Write** tab, delete the long welcome message and write a few words about what you fixed. -- Click **Create Pull Request**. - -That's it! Thank you for your contribution! Your pull request will be added manually to the main Dataverse Project board at https://github.com/orgs/IQSS/projects/34 and will go through code review and QA before it is merged into the "develop" branch. Along the way, developers might suggest changes or make them on your behalf. Once your pull request has been merged you will be listed as a contributor at https://github.com/IQSS/dataverse/graphs/contributors - -Please see https://github.com/IQSS/dataverse/pull/5857 for an example of a quick fix that was merged (the "Files changed" tab shows how a typo was fixed). - -Preview your documentation changes which will be built automatically as part of your pull request in Github. It will show up as a check entitled: `docs/readthedocs.org:dataverse-guide — Read the Docs build succeeded!`. For example, this PR built to https://dataverse-guide--9249.org.readthedocs.build/en/9249/. - -If you would like to read more about the Dataverse Project's use of GitHub, please see the :doc:`version-control` section. For bug fixes and features we request that you create an issue before making a pull request but this is not at all necessary for quick fixes to the documentation. - -.. _admin: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/admin -.. _api: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/api -.. _developers: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/developers -.. _installation: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/installation -.. _style: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/style -.. _user: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/user - -Building the Guides with Sphinx -------------------------------- - -The Dataverse guides are written using Sphinx (https://sphinx-doc.org). We recommend installing Sphinx on your localhost or using a Sphinx Docker container to build the guides locally so you can get an accurate preview of your changes. - -In case you decide to use a Sphinx Docker container to build the guides, you can skip the next two installation sections, but you will need to have Docker installed. - -Installing Sphinx -~~~~~~~~~~~~~~~~~ - -First, make a fork of https://github.com/IQSS/dataverse and clone your fork locally. Then change to the ``doc/sphinx-guides`` directory. - -``cd doc/sphinx-guides`` - -Create a Python virtual environment, activate it, then install dependencies: - -``python3 -m venv venv`` - -``source venv/bin/activate`` - -``pip install -r requirements.txt`` - -Installing GraphViz -~~~~~~~~~~~~~~~~~~~ - -In some parts of the documentation, graphs are rendered as images using the Sphinx GraphViz extension. - -Building the guides requires the ``dot`` executable from GraphViz. - -This requires having `GraphViz `_ installed and either having ``dot`` on the path or -`adding options to the make call `_. - -Editing and Building the Guides -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To edit the existing documentation: - -- Create a branch (see :ref:`how-to-make-a-pull-request`). -- In ``doc/sphinx-guides/source`` you will find the .rst files that correspond to https://guides.dataverse.org. -- Using your preferred text editor, open and edit the necessary files, or create new ones. - -Once you are done, you can preview the changes by building the guides locally. As explained, you can build the guides with Sphinx locally installed, or with a Docker container. - -Building the Guides with Sphinx Locally Installed -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Open a terminal, change directories to ``doc/sphinx-guides``, activate (or reactivate) your Python virtual environment, and build the guides. - -``cd doc/sphinx-guides`` - -``source venv/bin/activate`` - -``make clean`` - -``make html`` - -Building the Guides with a Sphinx Docker Container -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you want to build the guides using a Docker container, execute the following command in the repository root: - -``docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make html"`` - -Previewing the Guides -^^^^^^^^^^^^^^^^^^^^^ - -After Sphinx is done processing the files you should notice that the ``html`` folder in ``doc/sphinx-guides/build`` directory has been updated. -You can click on the files in the ``html`` folder to preview the changes. - -Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See :ref:`how-to-make-a-pull-request`. - -Versions --------- - -For installations hosting their own copies of the guides, note that as each version of the Dataverse Software is released, there is an updated version of the guides released with it. Google and other search engines index all versions, which may confuse users who discover your guides in the search results as to which version they should be looking at. When learning about your installation from the search results, it is best to be viewing the *latest* version. - -In order to make it clear to the crawlers that we only want the latest version discoverable in their search results, we suggest adding this to your ``robots.txt`` file:: - - User-agent: * - Allow: /en/latest/ - Disallow: /en/ - -PDF Version of the Guides -------------------------- - -The HTML version of the guides is the official one. Any other formats are maintained on a best effort basis. - -If you would like to build a PDF version of the guides and have Docker installed, please try the command below from the root of the git repo: - -``docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"`` - -A few notes about the command above: - -- Hopefully the PDF was created at ``doc/sphinx-guides/build/latex/Dataverse.pdf``. -- For now, we are using "nonstopmode" but this masks some errors. -- See requirements.txt for a note regarding the version of Sphinx we are using. - -Also, as of this writing we have enabled PDF builds from the "develop" branch. You download the PDF from http://preview.guides.gdcc.io/_/downloads/en/develop/pdf/ - -If you would like to help improve the PDF version of the guides, please get in touch! Please see :ref:`getting-help-developers` for ways to contact the developer community. diff --git a/doc/sphinx-guides/source/developers/index.rst b/doc/sphinx-guides/source/developers/index.rst index 25fea138736..6d01e13d78e 100755 --- a/doc/sphinx-guides/source/developers/index.rst +++ b/doc/sphinx-guides/source/developers/index.rst @@ -18,7 +18,6 @@ Developer Guide version-control sql-upgrade-scripts testing - documentation api-design security performance From f2871102a9b5a5c4d8dc5d97938f9b876f3ca636 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 10:53:17 -0400 Subject: [PATCH 21/31] reorg into writing guidelines #10531 --- doc/sphinx-guides/source/contributor/documentation.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index e0b83560870..37d171a4279 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -122,7 +122,9 @@ Also, as of this writing we have enabled PDF builds from the "develop" branch. Y If you would like to help improve the PDF version of the guides, please get in touch! Please see {ref}`getting-help-developers` for ways to contact the developer community. -## Writing Style Guidelines +## Writing Guidelines + +### Writing Style Guidelines Please observe the following when writing documentation: @@ -132,7 +134,7 @@ Please observe the following when writing documentation: - Use "double quotes" instead of 'single quotes'. - Favor "and" (data and code) over slashes (data/code). -## Table of Contents +### Table of Contents Every non-index page should use the following code to display a table of contents of internal sub-headings. This code should be placed below any introductory text and directly above the first subheading, much like a Wikipedia page. @@ -148,7 +150,7 @@ If the page is written in Markdown (.md), use this form: :depth: 3 ``` -## Images +### Images A good documentation is just like a website enhanced and upgraded by adding high quality and self-explanatory images. Often images depict a lot of written text in a simple manner. Within our Sphinx docs, you can add them in two ways: a) add a PNG image directly and include or b) use inline description languages like GraphViz (current only option). @@ -156,6 +158,6 @@ While PNGs in the git repo can be linked directly via URL, Sphinx-generated imag TODO: The above covers "how" but what about "when"? -## Cross References +### Cross References When adding ReStructured Text (.rst) [cross references](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role), use the hyphen character (`-`) as the word separator for the cross reference label. For example, `my-reference-label` would be the preferred label for a cross reference as opposed to, for example, `my_reference_label`. From 15c02842f083761bac1a56f2cd96c4f95c4f19a2 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 11:56:27 -0400 Subject: [PATCH 22/31] add intro for writing docs #10531 --- .../source/contributor/documentation.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 37d171a4279..3df81dc461c 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -7,6 +7,23 @@ Thank you for your interest in contributing documentation to Dataverse! Good doc :depth: 3 ``` +## Overview + +The Dataverse guides are written using [Sphinx](https://sphinx-doc.org). + +The source files are stored under [doc/sphinx-guides](https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides) in the main "dataverse" repo on GitHub. + +Historically, guides have been written in the default Sphinx format, [reStructuredText][] (.rst), but newer guides such as the {doc}`/contributor/index` are written in [Markdown][] (.md). + +[reStructuredText]: https://en.wikipedia.org/wiki/ReStructuredText +[Markdown]: https://en.wikipedia.org/wiki/Markdown + +Below we'll present a technique for making quick edits to the guides using GitHub's web editor ("quick fix"). We'll also describe how to install Sphinx locally for more significant edits. + +Finally, we'll provide some guidelines on writing content. + +We could use some help on writing this very page and helping newcomers get started. Please don't be shy about suggesting improvements! You can open an issue at , post to , write to the [mailing list](https://groups.google.com/g/dataverse-community), or suggest a change with a pull request. + ## Quick Fix If you find a typo or a small error in the documentation you can fix it using GitHub's online web editor. Generally speaking, we will be following @@ -33,7 +50,7 @@ If you would like to read more about the Dataverse Project's use of GitHub, plea ## Building the Guides with Sphinx -The Dataverse guides are written using Sphinx (). We recommend installing Sphinx on your computer or using a Sphinx Docker container to build the guides locally so you can get an accurate preview of your changes. +While the "quick fix" technique shown above should work fine for minor changes, especially for larger changes, we recommend installing Sphinx on your computer or using a Sphinx Docker container to build the guides locally so you can get an accurate preview of your changes. In case you decide to use a Sphinx Docker container to build the guides, you can skip the next two installation sections, but you will need to have Docker installed. From d92cb686fe9387c36d6557db541aaf76b88f8139 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 12:23:47 -0400 Subject: [PATCH 23/31] rewrite "versions" content into "hosting own guides" #10531 --- .../source/contributor/documentation.md | 24 +++++++++++-------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 3df81dc461c..8c1bce3f31c 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -111,16 +111,6 @@ After Sphinx is done processing the files you should notice that the `html` fold Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See {ref}`how-to-make-a-pull-request`. -## Versions - -For installations hosting their own copies of the guides, note that as each version of the Dataverse Software is released, there is an updated version of the guides released with it. Google and other search engines index all versions, which may confuse users who discover your guides in the search results as to which version they should be looking at. When learning about your installation from the search results, it is best to be viewing the *latest* version. - -In order to make it clear to the crawlers that we only want the latest version discoverable in their search results, we suggest adding this to your `robots.txt` file: - - User-agent: * - Allow: /en/latest/ - Disallow: /en/ - ## PDF Version of the Guides The HTML version of the guides is the official one. Any other formats are maintained on a best effort basis. @@ -178,3 +168,17 @@ TODO: The above covers "how" but what about "when"? ### Cross References When adding ReStructured Text (.rst) [cross references](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role), use the hyphen character (`-`) as the word separator for the cross reference label. For example, `my-reference-label` would be the preferred label for a cross reference as opposed to, for example, `my_reference_label`. + +## Hosting Your Own Version of the Guides + +Some installations of Dataverse maintain their own versions of the guides and use settings like {ref}`:NavbarGuidesUrl` or {ref}`:GuidesBaseUrl` to point their users to them. + +### Having Google Index the Latest Version + +As each version of the Dataverse software is released, there is an updated version of the guides released with it. Google and other search engines index all versions, which may confuse users who discover your guides in the search results as to which version they should be looking at. When learning about your installation from the search results, it is best to be viewing the *latest* version. + +In order to make it clear to the crawlers that we only want the latest version discoverable in their search results, we suggest adding this to your `robots.txt` file: + + User-agent: * + Allow: /en/latest/ + Disallow: /en/ From 6b3d06ceaed8013f9c70741d4714d6750906d686 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 12:25:40 -0400 Subject: [PATCH 24/31] move PDF content down (less important) #10531 --- .../source/contributor/documentation.md | 37 ++++++++++--------- 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 8c1bce3f31c..0c4955d3e5e 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -111,24 +111,6 @@ After Sphinx is done processing the files you should notice that the `html` fold Now you can make a commit with the changes to your own fork in GitHub and submit a pull request. See {ref}`how-to-make-a-pull-request`. -## PDF Version of the Guides - -The HTML version of the guides is the official one. Any other formats are maintained on a best effort basis. - -If you would like to build a PDF version of the guides and have Docker installed, please try the command below from the root of the git repo: - -`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"` - -A few notes about the command above: - -- Hopefully the PDF was created at `doc/sphinx-guides/build/latex/Dataverse.pdf`. -- For now, we are using "nonstopmode" but this masks some errors. -- See requirements.txt for a note regarding the version of Sphinx we are using. - -Also, as of this writing we have enabled PDF builds from the "develop" branch. You download the PDF from - -If you would like to help improve the PDF version of the guides, please get in touch! Please see {ref}`getting-help-developers` for ways to contact the developer community. - ## Writing Guidelines ### Writing Style Guidelines @@ -169,6 +151,25 @@ TODO: The above covers "how" but what about "when"? When adding ReStructured Text (.rst) [cross references](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role), use the hyphen character (`-`) as the word separator for the cross reference label. For example, `my-reference-label` would be the preferred label for a cross reference as opposed to, for example, `my_reference_label`. +## PDF Version of the Guides + +The HTML version of the guides is the official one. Any other formats are maintained on a best effort basis. + +If you would like to build a PDF version of the guides and have Docker installed, please try the command below from the root of the git repo: + +`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"` + +A few notes about the command above: + +- Hopefully the PDF was created at `doc/sphinx-guides/build/latex/Dataverse.pdf`. +- For now, we are using "nonstopmode" but this masks some errors. +- See requirements.txt for a note regarding the version of Sphinx we are using. + +Also, as of this writing we have enabled PDF builds from the "develop" branch. You download the PDF from + +If you would like to help improve the PDF version of the guides, please get in touch! Please see {ref}`getting-help-developers` for ways to contact the developer community. + + ## Hosting Your Own Version of the Guides Some installations of Dataverse maintain their own versions of the guides and use settings like {ref}`:NavbarGuidesUrl` or {ref}`:GuidesBaseUrl` to point their users to them. From 147866b92725bc38422ee89c44ededa158ea1a1e Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 12:33:09 -0400 Subject: [PATCH 25/31] add release note #10531 --- doc/release-notes/10531-contrib.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/release-notes/10531-contrib.md diff --git a/doc/release-notes/10531-contrib.md b/doc/release-notes/10531-contrib.md new file mode 100644 index 00000000000..6cfbe988992 --- /dev/null +++ b/doc/release-notes/10531-contrib.md @@ -0,0 +1 @@ +A new [Contributor Guide](https://dataverse-guide--10532.org.readthedocs.build/en/10532/contributor/index.html) has been added by the UX Working Group (#10531 and #10532). From 5c5e9f7fafb9b3a46dd3dc1c3988ee0018d947d6 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Thu, 13 Jun 2024 12:59:50 -0400 Subject: [PATCH 26/31] tweaks #10531 --- doc/sphinx-guides/source/contributor/index.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index e603d6151b7..7a9555fdd34 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -1,6 +1,6 @@ # Contributor Guide -Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. We welcome contributions of ideas, bug reports, usability research/feedback, documentation, code, and more! +Thank you for your interest in contributing to Dataverse! We are open to contributions from everyone. We welcome contributions of ideas, bug reports, documentation, code, and more! ```{contents} Contents: :local: @@ -68,9 +68,11 @@ Consider contributing to , a git ## Issue Triage -New issues are coming in all the time, especially for the main issue tracker at . +New issues come in all the time, especially for the main issue tracker at . -You can help by leaving comments that mention related issues or that answer questions. +You can help by leaving comments. You can mention related issues or answer questions. + +If you are interested in adding issue labels or related curation, please get in touch! ## Giving Talks @@ -82,13 +84,15 @@ For non-recorded talks, we are happy to upload your slides to . +You're welcome to start your own working group, of course. We can help you get the word out. + ## Artwork -Surely we can put artistic talent to use. See for a story about drawing animals chatting about Dataverse. +Surely we can put artistic talent to use. A contributor [drew cartoon animals chatting about Dataverse](https://github.com/IQSS/chat.dataverse.org/issues/18), for example. As [annnounced](https://groups.google.com/g/dataverse-community/c/pM39_9O5Rug/m/CK-gJqZFBgAJ), we have a [sticker template](https://dataverse.org/sites/projects.iq.harvard.edu/files/dataverseorg/files/dataverse_community_stickers_template.zip) you can use. -See [Illustrations from The Turing Way](https://zenodo.org/doi/10.5281/zenodo.3332807) for how that community has created artwork. Perhaps we can do the same. +See [Illustrations from The Turing Way](https://zenodo.org/doi/10.5281/zenodo.3332807) for how that community has created artwork. Perhaps we can create a similar collection. ## Other Contributions From 946d1ddd75e24d053ddd0ed0ad4e0b4b848b0fd8 Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Fri, 14 Jun 2024 11:49:35 -0400 Subject: [PATCH 27/31] incorporate feedback from @philippconzett #10531 Via https://groups.google.com/g/dataverse-community/c/ejJrJmH6ojI/m/NnLSTlaMDwAJ --- doc/sphinx-guides/source/contributor/code.md | 2 +- doc/sphinx-guides/source/contributor/documentation.md | 10 +++++----- doc/sphinx-guides/source/contributor/index.md | 4 ++++ 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/code.md b/doc/sphinx-guides/source/contributor/code.md index 9bc2b5a2027..2a1dec08c05 100644 --- a/doc/sphinx-guides/source/contributor/code.md +++ b/doc/sphinx-guides/source/contributor/code.md @@ -45,7 +45,7 @@ For other codebases, consult the README. ## Reviewing Code -Reviewing code is a great way to learn about a codebase. For any codebase you can browse open pull requests, of course, but for the primary codebases, you can take a look at the "Ready for Review" and "In Review" columns at https://github.com/orgs/IQSS/projects/34 +Reviewing code is a great way to learn about a codebase. For any codebase you can browse open pull requests, of course, but for the primary codebases, you can take a look at the "Ready for Review" and "In Review" columns at . You are welcome to review code informally or to leave an actual review. We're interested in what you think. diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 0c4955d3e5e..67db855dc60 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -1,6 +1,6 @@ # Writing Documentation -Thank you for your interest in contributing documentation to Dataverse! Good documentation is absolutely critical to the success of a software project. +Thank you for your interest in contributing documentation to Dataverse! Good documentation is absolutely critical to the success of software. ```{contents} Contents: :local: @@ -26,23 +26,23 @@ We could use some help on writing this very page and helping newcomers get start ## Quick Fix -If you find a typo or a small error in the documentation you can fix it using GitHub's online web editor. Generally speaking, we will be following +If you find a typo or a small error in the documentation you can fix it using GitHub's online web editor. Generally speaking, we will be following [GitHub's guidance on editing files in another user's repository](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files#editing-files-in-another-users-repository). - Navigate to where you will see folders for each of the guides: [admin][], [api][], [container][], etc. - Find the file you want to edit under one of the folders above. -- Click the pencil icon in the upper-right corner. If this is your first contribution to the Dataverse Project, the hover text over the pencil icon will say "Fork this project and edit this file". +- Click the pencil icon in the upper-right corner. If this is your first contribution to Dataverse, the hover text over the pencil icon will say "Fork this project and edit this file". - Make changes to the file and preview them. - In the **Commit changes** box, enter a description of the changes you have made and click **Propose file change**. - Under the **Write** tab, delete the long welcome message and write a few words about what you fixed. - Click **Create Pull Request**. -That's it! Thank you for your contribution! Your pull request will be added manually to the main Dataverse Project board at and will go through code review and QA before it is merged into the "develop" branch. Along the way, developers might suggest changes or make them on your behalf. Once your pull request has been merged you will be listed as a contributor at ! 🎉 +That's it! Thank you for your contribution! Your pull request will be added manually to the main Dataverse project board at and will go through code review and QA before it is merged into the "develop" branch. Along the way, developers might suggest changes or make them on your behalf. Once your pull request has been merged you will be listed as a contributor at ! 🎉 Please see for an example of a quick fix that was merged (the "Files changed" tab shows how a typo was fixed). Preview your documentation changes which will be built automatically as part of your pull request in Github. It will show up as a check entitled: `docs/readthedocs.org:dataverse-guide — Read the Docs build succeeded!`. For example, this PR built to . -If you would like to read more about the Dataverse Project's use of GitHub, please see the {doc}`/developers/version-control` section of the Developer Guide. For bug fixes and features we request that you create an issue before making a pull request but this is not at all necessary for quick fixes to the documentation. +If you would like to read more about the Dataverse's use of GitHub, please see the {doc}`/developers/version-control` section of the Developer Guide. For bug fixes and features we request that you create an issue before making a pull request but this is not at all necessary for quick fixes to the documentation. [admin]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/admin [api]: https://github.com/IQSS/dataverse/tree/develop/doc/sphinx-guides/source/api diff --git a/doc/sphinx-guides/source/contributor/index.md b/doc/sphinx-guides/source/contributor/index.md index 7a9555fdd34..e75cc58bccd 100644 --- a/doc/sphinx-guides/source/contributor/index.md +++ b/doc/sphinx-guides/source/contributor/index.md @@ -86,6 +86,10 @@ Most working groups are wide open to participation. For the current list of grou You're welcome to start your own working group, of course. We can help you get the word out. +## GDCC + +The popularity of the Dataverse software has resulted in a continuously growing community with different needs and requirements. The Global Dataverse Community Consortium (GDCC) helps coordinate community efforts and sustain the software and community in the long term. Please consider contributing to GDCC by joining as an institutional member (). + ## Artwork Surely we can put artistic talent to use. A contributor [drew cartoon animals chatting about Dataverse](https://github.com/IQSS/chat.dataverse.org/issues/18), for example. From 9bd0ccff917a94eb76a75aacffa71d8a9d42fd8b Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Mon, 1 Jul 2024 12:13:23 -0400 Subject: [PATCH 28/31] re-apply changes about sphinx and docker #10531 These changes were in 853965e as part of PR #10565 but were lost during resolution of a merge conflict. --- .../source/contributor/documentation.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 67db855dc60..6a24e653152 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -87,7 +87,7 @@ To edit the existing documentation: Once you are done, you can preview the changes by building the guides locally. As explained, you can build the guides with Sphinx locally installed, or with a Docker container. -#### Building the Guides with Sphinx Locally Installed +#### Building the Guides with Sphinx Installed Locally Open a terminal, change directories to `doc/sphinx-guides`, activate (or reactivate) your Python virtual environment, and build the guides. @@ -99,11 +99,20 @@ Open a terminal, change directories to `doc/sphinx-guides`, activate (or reactiv `make html` -#### Building the Guides with a Sphinx Docker Container +#### Building the Guides with a Sphinx Docker Container and a Makefile + +We have added a Makefile to simplify the process of building the guides using a Docker container, you can use the following commands from the repository root: + +- `make docs-html` +- `make docs-pdf` +- `make docs-epub` +- `make docs-all` + +#### Building the Guides with a Sphinx Docker Container and CLI If you want to build the guides using a Docker container, execute the following command in the repository root: -`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make html"` +`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx:7.2.6 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make html"` #### Previewing the Guides @@ -157,7 +166,7 @@ The HTML version of the guides is the official one. Any other formats are mainta If you would like to build a PDF version of the guides and have Docker installed, please try the command below from the root of the git repo: -`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:3.5.4 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"` +`docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx-latexpdf:7.2.6 bash -c "cd doc/sphinx-guides && pip3 install -r requirements.txt && make latexpdf LATEXMKOPTS=\"-interaction=nonstopmode\"; cd ../.. && ls -1 doc/sphinx-guides/build/latex/Dataverse.pdf"` A few notes about the command above: From 7a64dea885c6fcaf71e654a4be9e920dc21c3bab Mon Sep 17 00:00:00 2001 From: Philip Durbin Date: Mon, 1 Jul 2024 12:15:32 -0400 Subject: [PATCH 29/31] remove TODO about how vs when #10531 --- doc/sphinx-guides/source/contributor/documentation.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/sphinx-guides/source/contributor/documentation.md b/doc/sphinx-guides/source/contributor/documentation.md index 6a24e653152..12a4266c9ff 100644 --- a/doc/sphinx-guides/source/contributor/documentation.md +++ b/doc/sphinx-guides/source/contributor/documentation.md @@ -154,8 +154,6 @@ A good documentation is just like a website enhanced and upgraded by adding high While PNGs in the git repo can be linked directly via URL, Sphinx-generated images do not need a manual step and might provide higher visual quality. Especially in terms of quality of content, generated images can be extendend and improved by a textbased and reviewable commit, without needing raw data or source files and no diff around. -TODO: The above covers "how" but what about "when"? - ### Cross References When adding ReStructured Text (.rst) [cross references](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#ref-role), use the hyphen character (`-`) as the word separator for the cross reference label. For example, `my-reference-label` would be the preferred label for a cross reference as opposed to, for example, `my_reference_label`. From dddcf29188a5c35174f3c94ffc1c4cb1d7fc0552 Mon Sep 17 00:00:00 2001 From: Don Sizemore Date: Mon, 1 Jul 2024 13:23:46 -0400 Subject: [PATCH 30/31] bump AWS SDK to current version (#10648) Co-authored-by: Don Sizemore --- modules/dataverse-parent/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/dataverse-parent/pom.xml b/modules/dataverse-parent/pom.xml index f286a44841f..4a67e301f5b 100644 --- a/modules/dataverse-parent/pom.xml +++ b/modules/dataverse-parent/pom.xml @@ -151,7 +151,7 @@ 6.2024.6 42.7.2 9.4.1 - 1.12.290 + 1.12.748 26.30.0 From b42222fd35006c24e57eccece81d3496dcac31b8 Mon Sep 17 00:00:00 2001 From: Guillermo Portas Date: Tue, 2 Jul 2024 15:59:19 +0100 Subject: [PATCH 31/31] Fix metadata field type display condition in dataverses/{id}/metadatablocks API endpoint (#10642) * Fixed: metadata field type display condition in dataverses/{id}/metadatablocks * Changed: json object builder instantiation * Added: extended test coverage for testUpdateInputLevels and testFeatureDataverse * Added: release notes for #10637 * Fixed: JsonPrinter metadata blocks dataset field type isRequired logic * Refactor: simpler conditions in jsonPrinter * Refactor: reordered condition in jsonPrinter * Fixed: displayCondition in jsonPrinter --- .../10637-fix-dataverse-metadatablocks-api.md | 2 + .../edu/harvard/iq/dataverse/Dataverse.java | 18 ++++--- .../iq/dataverse/util/json/JsonPrinter.java | 18 +++++-- .../iq/dataverse/api/DataversesIT.java | 52 +++++++++++++------ .../edu/harvard/iq/dataverse/api/UtilIT.java | 21 ++++---- 5 files changed, 76 insertions(+), 35 deletions(-) create mode 100644 doc/release-notes/10637-fix-dataverse-metadatablocks-api.md diff --git a/doc/release-notes/10637-fix-dataverse-metadatablocks-api.md b/doc/release-notes/10637-fix-dataverse-metadatablocks-api.md new file mode 100644 index 00000000000..c8c9c4fa66f --- /dev/null +++ b/doc/release-notes/10637-fix-dataverse-metadatablocks-api.md @@ -0,0 +1,2 @@ +dataverses/{id}/metadatablocks API endpoint has been fixed, since the fields returned for each metadata block when returnDatasetTypes query parameter is set to true was not correct. + diff --git a/src/main/java/edu/harvard/iq/dataverse/Dataverse.java b/src/main/java/edu/harvard/iq/dataverse/Dataverse.java index 78b1827c798..978c716e058 100644 --- a/src/main/java/edu/harvard/iq/dataverse/Dataverse.java +++ b/src/main/java/edu/harvard/iq/dataverse/Dataverse.java @@ -412,12 +412,18 @@ public List getDataverseFieldTypeInputLevels() { } public boolean isDatasetFieldTypeRequiredAsInputLevel(Long datasetFieldTypeId) { - for(DataverseFieldTypeInputLevel dataverseFieldTypeInputLevel : dataverseFieldTypeInputLevels) { - if (dataverseFieldTypeInputLevel.getDatasetFieldType().getId().equals(datasetFieldTypeId) && dataverseFieldTypeInputLevel.isRequired()) { - return true; - } - } - return false; + return dataverseFieldTypeInputLevels.stream() + .anyMatch(inputLevel -> inputLevel.getDatasetFieldType().getId().equals(datasetFieldTypeId) && inputLevel.isRequired()); + } + + public boolean isDatasetFieldTypeIncludedAsInputLevel(Long datasetFieldTypeId) { + return dataverseFieldTypeInputLevels.stream() + .anyMatch(inputLevel -> inputLevel.getDatasetFieldType().getId().equals(datasetFieldTypeId) && inputLevel.isInclude()); + } + + public boolean isDatasetFieldTypeInInputLevels(Long datasetFieldTypeId) { + return dataverseFieldTypeInputLevels.stream() + .anyMatch(inputLevel -> inputLevel.getDatasetFieldType().getId().equals(datasetFieldTypeId)); } public Template getDefaultTemplate() { diff --git a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java index 95f14b79ece..c72dfc1d127 100644 --- a/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java +++ b/src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java @@ -640,18 +640,26 @@ public static JsonObjectBuilder json(MetadataBlock metadataBlock, boolean printO JsonObjectBuilder fieldsBuilder = Json.createObjectBuilder(); Set datasetFieldTypes = new TreeSet<>(metadataBlock.getDatasetFieldTypes()); + for (DatasetFieldType datasetFieldType : datasetFieldTypes) { - boolean requiredInOwnerDataverse = ownerDataverse != null && ownerDataverse.isDatasetFieldTypeRequiredAsInputLevel(datasetFieldType.getId()); - boolean displayCondition = !printOnlyDisplayedOnCreateDatasetFieldTypes || - datasetFieldType.isDisplayOnCreate() || - requiredInOwnerDataverse; + Long datasetFieldTypeId = datasetFieldType.getId(); + boolean requiredAsInputLevelInOwnerDataverse = ownerDataverse != null && ownerDataverse.isDatasetFieldTypeRequiredAsInputLevel(datasetFieldTypeId); + boolean includedAsInputLevelInOwnerDataverse = ownerDataverse != null && ownerDataverse.isDatasetFieldTypeIncludedAsInputLevel(datasetFieldTypeId); + boolean isNotInputLevelInOwnerDataverse = ownerDataverse != null && !ownerDataverse.isDatasetFieldTypeInInputLevels(datasetFieldTypeId); + + DatasetFieldType parentDatasetFieldType = datasetFieldType.getParentDatasetFieldType(); + boolean isRequired = parentDatasetFieldType == null ? datasetFieldType.isRequired() : parentDatasetFieldType.isRequired(); + + boolean displayCondition = printOnlyDisplayedOnCreateDatasetFieldTypes + ? (datasetFieldType.isDisplayOnCreate() || isRequired || requiredAsInputLevelInOwnerDataverse) + : ownerDataverse == null || includedAsInputLevelInOwnerDataverse || isNotInputLevelInOwnerDataverse; + if (displayCondition) { fieldsBuilder.add(datasetFieldType.getName(), json(datasetFieldType, ownerDataverse)); } } jsonObjectBuilder.add("fields", fieldsBuilder); - return jsonObjectBuilder; } diff --git a/src/test/java/edu/harvard/iq/dataverse/api/DataversesIT.java b/src/test/java/edu/harvard/iq/dataverse/api/DataversesIT.java index b072a803aa4..79cc46cfa79 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/DataversesIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/DataversesIT.java @@ -702,8 +702,10 @@ public void testListMetadataBlocks() { Response setMetadataBlocksResponse = UtilIT.setMetadataBlocks(dataverseAlias, Json.createArrayBuilder().add("citation").add("astrophysics"), apiToken); setMetadataBlocksResponse.then().assertThat().statusCode(OK.getStatusCode()); - String[] testInputLevelNames = {"geographicCoverage", "country"}; - Response updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, apiToken); + String[] testInputLevelNames = {"geographicCoverage", "country", "city"}; + boolean[] testRequiredInputLevels = {false, true, false}; + boolean[] testIncludedInputLevels = {false, true, true}; + Response updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, testRequiredInputLevels, testIncludedInputLevels, apiToken); updateDataverseInputLevelsResponse.then().assertThat().statusCode(OK.getStatusCode()); // Dataverse not found @@ -769,6 +771,21 @@ public void testListMetadataBlocks() { assertThat(expectedAllMetadataBlockDisplayNames, hasItemInArray(actualMetadataBlockDisplayName2)); assertThat(expectedAllMetadataBlockDisplayNames, hasItemInArray(actualMetadataBlockDisplayName3)); + // Check dataset fields for the updated input levels are retrieved + int geospatialMetadataBlockIndex = actualMetadataBlockDisplayName1.equals("Geospatial Metadata") ? 0 : actualMetadataBlockDisplayName2.equals("Geospatial Metadata") ? 1 : 2; + + // Since the included property of geographicCoverage is set to false, we should retrieve the total number of fields minus one + listMetadataBlocksResponse.then().assertThat() + .body(String.format("data[%d].fields.size()", geospatialMetadataBlockIndex), equalTo(10)); + + String actualMetadataField1 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.geographicCoverage.name", geospatialMetadataBlockIndex)); + String actualMetadataField2 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.country.name", geospatialMetadataBlockIndex)); + String actualMetadataField3 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.city.name", geospatialMetadataBlockIndex)); + + assertNull(actualMetadataField1); + assertNotNull(actualMetadataField2); + assertNotNull(actualMetadataField3); + // Existent dataverse and onlyDisplayedOnCreate=true and returnDatasetFieldTypes=true listMetadataBlocksResponse = UtilIT.listMetadataBlocks(dataverseAlias, true, true, apiToken); listMetadataBlocksResponse.then().assertThat().statusCode(OK.getStatusCode()); @@ -785,16 +802,18 @@ public void testListMetadataBlocks() { assertThat(expectedOnlyDisplayedOnCreateMetadataBlockDisplayNames, hasItemInArray(actualMetadataBlockDisplayName2)); // Check dataset fields for the updated input levels are retrieved - int geospatialMetadataBlockIndex = actualMetadataBlockDisplayName2.equals("Geospatial Metadata") ? 1 : 0; + geospatialMetadataBlockIndex = actualMetadataBlockDisplayName2.equals("Geospatial Metadata") ? 1 : 0; listMetadataBlocksResponse.then().assertThat() - .body(String.format("data[%d].fields.size()", geospatialMetadataBlockIndex), equalTo(2)); + .body(String.format("data[%d].fields.size()", geospatialMetadataBlockIndex), equalTo(1)); - String actualMetadataField1 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.geographicCoverage.name", geospatialMetadataBlockIndex)); - String actualMetadataField2 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.country.name", geospatialMetadataBlockIndex)); + actualMetadataField1 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.geographicCoverage.name", geospatialMetadataBlockIndex)); + actualMetadataField2 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.country.name", geospatialMetadataBlockIndex)); + actualMetadataField3 = listMetadataBlocksResponse.then().extract().path(String.format("data[%d].fields.city.name", geospatialMetadataBlockIndex)); - assertNotNull(actualMetadataField1); + assertNull(actualMetadataField1); assertNotNull(actualMetadataField2); + assertNull(actualMetadataField3); // User has no permissions on the requested dataverse Response createSecondUserResponse = UtilIT.createRandomUser(); @@ -898,12 +917,16 @@ public void testUpdateInputLevels() { // Update valid input levels String[] testInputLevelNames = {"geographicCoverage", "country"}; - Response updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, apiToken); + boolean[] testRequiredInputLevels = {true, false}; + boolean[] testIncludedInputLevels = {true, false}; + Response updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, testRequiredInputLevels, testIncludedInputLevels, apiToken); + String actualInputLevelName = updateDataverseInputLevelsResponse.then().extract().path("data.inputLevels[0].datasetFieldTypeName"); + int geographicCoverageInputLevelIndex = actualInputLevelName.equals("geographicCoverage") ? 0 : 1; updateDataverseInputLevelsResponse.then().assertThat() - .body("data.inputLevels[0].required", equalTo(true)) - .body("data.inputLevels[0].include", equalTo(true)) - .body("data.inputLevels[1].required", equalTo(true)) - .body("data.inputLevels[1].include", equalTo(true)) + .body(String.format("data.inputLevels[%d].include", geographicCoverageInputLevelIndex), equalTo(true)) + .body(String.format("data.inputLevels[%d].required", geographicCoverageInputLevelIndex), equalTo(true)) + .body(String.format("data.inputLevels[%d].include", 1 - geographicCoverageInputLevelIndex), equalTo(false)) + .body(String.format("data.inputLevels[%d].required", 1 - geographicCoverageInputLevelIndex), equalTo(false)) .statusCode(OK.getStatusCode()); String actualFieldTypeName1 = updateDataverseInputLevelsResponse.then().extract().path("data.inputLevels[0].datasetFieldTypeName"); String actualFieldTypeName2 = updateDataverseInputLevelsResponse.then().extract().path("data.inputLevels[1].datasetFieldTypeName"); @@ -913,15 +936,14 @@ public void testUpdateInputLevels() { // Update input levels with an invalid field type name String[] testInvalidInputLevelNames = {"geographicCoverage", "invalid1"}; - updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInvalidInputLevelNames, apiToken); + updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInvalidInputLevelNames, testRequiredInputLevels, testIncludedInputLevels, apiToken); updateDataverseInputLevelsResponse.then().assertThat() .body("message", equalTo("Invalid dataset field type name: invalid1")) .statusCode(BAD_REQUEST.getStatusCode()); // Update invalid empty input levels testInputLevelNames = new String[]{}; - updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, apiToken); - updateDataverseInputLevelsResponse.prettyPrint(); + updateDataverseInputLevelsResponse = UtilIT.updateDataverseInputLevels(dataverseAlias, testInputLevelNames, testRequiredInputLevels, testIncludedInputLevels, apiToken); updateDataverseInputLevelsResponse.then().assertThat() .body("message", equalTo("Error while updating dataverse input levels: Input level list cannot be null or empty")) .statusCode(INTERNAL_SERVER_ERROR.getStatusCode()); diff --git a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java index 257610dbc32..0216859b869 100644 --- a/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java +++ b/src/test/java/edu/harvard/iq/dataverse/api/UtilIT.java @@ -3960,22 +3960,25 @@ static Response requestGlobusUploadPaths(Integer datasetId, JsonObject body, Str .post("/api/datasets/" + datasetId + "/requestGlobusUploadPaths"); } - static Response updateDataverseInputLevels(String dataverseAlias, String[] inputLevelNames, String apiToken) { - JsonArrayBuilder contactArrayBuilder = Json.createArrayBuilder(); - for(String inputLevelName : inputLevelNames) { - contactArrayBuilder.add(Json.createObjectBuilder() - .add("datasetFieldTypeName", inputLevelName) - .add("required", true) - .add("include", true) - ); + public static Response updateDataverseInputLevels(String dataverseAlias, String[] inputLevelNames, boolean[] requiredInputLevels, boolean[] includedInputLevels, String apiToken) { + JsonArrayBuilder inputLevelsArrayBuilder = Json.createArrayBuilder(); + for (int i = 0; i < inputLevelNames.length; i++) { + inputLevelsArrayBuilder.add(createInputLevelObject(inputLevelNames[i], requiredInputLevels[i], includedInputLevels[i])); } return given() .header(API_TOKEN_HTTP_HEADER, apiToken) - .body(contactArrayBuilder.build().toString()) + .body(inputLevelsArrayBuilder.build().toString()) .contentType(ContentType.JSON) .put("/api/dataverses/" + dataverseAlias + "/inputLevels"); } + private static JsonObjectBuilder createInputLevelObject(String name, boolean required, boolean include) { + return Json.createObjectBuilder() + .add("datasetFieldTypeName", name) + .add("required", required) + .add("include", include); + } + public static Response getOpenAPI(String accept, String format) { Response response = given() .header("Accept", accept)