Support for configurable list of licenses

[janvanmansum]

This PR is currently an implementation of the "Consensus Proposal" from this document: https://docs.google.com/document/d/10htygglMdlABYWqtcZpqd8sHOwIe6sLL_UJtTv8NEKw/edit . Key parts of this document are replicated in the fields below.

What this PR does / why we need it:

Current State:
Dataverse allows only either CC0 or Custom Terms to be specified for a Dataset. There is broad interest in supporting additional licenses in a managed, machine readable way.

Consensus Proposal:
As part of its efforts, DANS has worked with IQSS and GDCC to define an update to support multiple licenses that it will implement and contribute to the Dataverse community.

Summary:
The proposed work will make it possible for site administrators to configure their Dataverse installation with additional/alternate pre-defined license choices. Admins will also be able to decide whether custom licenses can be created (i.e. by adding entries for some of the existing entries on the Terms tab), whether there is a default license and, if so, which license it is. Users will be able to select from a list of configured licenses prior to publishing a Dataset version or enter custom terms (if custom licenses are allowed). The chosen license will also be more visible to both Dataset creators and those browsing or downloading from a Dataset– i.e. on the dataset page, in the publish dialog.

Which issue(s) this PR closes:

Closes #7440

Special notes for your reviewer:
The design of this feature has been changed since #7440 was described. For an up-to-date description of the feature as it has been implemented, see below test scenario and "Additional Documentation".

Suggestions on how to test this:

1. Build the .war file from this branch.

2. Install Dataverse release v5.5

3. Deploy the .war file you built in step 1: /usr/local/payara5/glassfish/bin/asadmin deploy --force --name dataverse /path/to/your-dataverse.war

4. Create a couple of license configuration JSON files, for example:

   • cc-by.json:

     {
       "uri": "http://creativecommons.org/licenses/by/4.0",
       "name": "CC-BY-4.0",
       "shortDescription": "Creative Commons Attribution 4.0 International License.",
       "iconUrl": "https://i.creativecommons.org/l/by/4.0/88x31.png",
       "active": true
     }

   • cc-by-sa.json:

     {
        "uri": "http://creativecommons.org/licenses/by-sa/4.0",
        "name": "CC-BY-SA-4.0",
        "shortDescription": "Creative Commons Attribution-ShareAlike 4.0 International License.",
        "iconUrl": "https://i.creativecommons.org/l/by-sa/4.0/88x31.png",
        "active": true
     }

5. Add the licenses to Dataverse:

   •  curl -X POST -H 'Content-Type: application/json' \ 
              http://localhost:8080/api/admin/licenses --data-binary @cc-by.json

   •  curl -X POST -H 'Content-Type: application/json' \
              http://localhost:8080/api/admin/licenses --data-binary @cc-by-sa.json    

6. Create a new dataset in Dataverse and navigate to the Terms tab. Click "Edit Terms Requirements". You should now have a drop-down to select from:
   

7. Turn off "Custom Terms" as follows:

   curl -X PUT --data false 'http://localhost:8080/api/admin/settings/:AllowCustomTerms'

8. Go back to the Terms tab and notice that the Custom Terms item is gone.

9. Select a license, save the dataset. The license is displayed in the desciption of the dataset:
   

10. Click Publish Dataset. The selected terms are displayed in the publish confirmation dialog:
    

11. Click Continue.

12. Further tests:

    • Inspect the metadata export to see the new license information there (if the format supports license information).
    • Turn :AllowCustomTerms on again. When selecting the "Custom Terms" item the "Dataset Terms" block is extended with the fields "Custom Terms" ... "Disclaimer".
    • Do a clean install of v5.5, create some datasets with Custom Terms and some with a CC0 waiver. Deploy the new .war file and check that the datasets still have the correct terms or CC0 waiver.
    • Update a license, setting active to false. It will no longer be available when editing datasets, but existing dataset(versions) are not affected.
    • Try to delete a license that is referenced by a dataset; this should be refused by the API.
    • Change the default license and create a new dataset, check that the default is pre-selected.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
See test scenarios in previous section.

Is there a release notes update needed for this change?:
No

Additional documentation:
See item "Manage Available Standard License Terms" in doc/sphinx-guides/source/api/native-api.rst.

[qqmyers]

2. Should be fixed by Retry updates DANS-KNAW/dataverse#144 - once that's merged, this PR should have the fix
3. I could add something like this to the backward compatibility issues list: - Rollback. In general one should not deploy an earlier release over a database that has been modified by deployment of a later release. (Make a db backup before upgrading and use that copy if you go back to a prior version.) Due to the nature of the db changes in this release, attempts to deploy an earlier version of Dataverse will fail unless the database is also restored to it's pre-release state.
   Let me know if that's what you want or if alternate text is needed prior to merge.

[kcondon]

@qqmyers Thanks, Jim. I think that added note on deploying would be helpful.

[qqmyers]

7. Fixed (as soon as PR is updated)
8. This works fine for me. Two things could be confusing: the default is true so the only time custom terms are not allowed is when that property is set to false. Since the Terms edit info is loaded with the dataset page, you have to refresh the page before the terms tab will update. If there's a repeatable problem, let me know. (There was a time when this was broken so it's possible that the PR branch is out of date, but I don't see any code differences.)
9. Previously Dataverse duplicated the license info (if set) and provided both a dcterms:license and dcterms:rights for CC0/CC0 Waiver, and only the dcterms:rights when custom terms were set. With the ability to have different licenses, the dcterms:license show the license name and the decision was made to drop the non-standard ~duplication of info. This is one of several related changes to license representation that the release notes attempt to indicate. If additional/alternate words are needed, please suggest some text.
10. Ah - of course line 1693 of dataset.xhtml was missing "DatasetPage." - now fixed as soon as PR updates (Good catch!). That phrase should only appear with custom terms and it looks like it was OK in the file downloads and terms tabs already.
12. "CC0 Waiver" was included in a non standard way in earlier Dataverse versions. Rather than include the license names (which would not be 'CC0 Waiver' for 'CC0 1.0') and continuing/altering the non-standard mechanism, we've opened Feature Request/Idea: Add license info to DDI exports #8355 per review discussions. (Also Support translation of license names and descriptions after #7920 #8346 to support i18n as the old hardcoded value of 'CC0 Waiver' in the Bundle which could be translated doesn't work for the multilicense case.)
13. The JSON export includes a license object with a name and URI instead. This was discussed in review and was added to the backward incompatibilities list
14. This was discussed in review. The new for should be parsable by Google ( the copy of json-ld in included in the header) and avoid a non-standard warning about the license object while also supporting multiple licenses. This change is noted in the incompatibilities list.

[kcondon]

@qqmyers OK, seems like 7 is still an issue so I suppose pr not updated yet? for 8, I had refreshed page, navigated away and back, and restarted payara. Will retest but I had tried all that. As for the export related stuff, 9, 11-14, I've asked Julian to take a look at your comments seeing as he is more up to speed on Design and Export. I am a bit confused by your ending comment on 10, unless you'd meant to make it 14. 10 is the popup text next to cc0 and you addressed it earlier.

[qqmyers]

Yeah - the raw text of my previous comment says 14 but the rendering changes that to 10. I made some edits above.

[qqmyers]

DANS-KNAW#145 is the one to watch for 7

[jggautier]

Hi all.

• We talked about the changes to the DDI exports and I saw the changes to the DC(Terms), JSON and Schema.org exports. I have questions about the DC(Terms) and Schema.org exports, but I'm not sure if those questions should hold up this PR or if it's okay to see how this works in practice before raising any issues.
• The release notes don't document the changes to the DDI and DC(Terms) exports, but I'm not sure if that's necessary. I interpret the release notes' "Backward Incompatibilities" section as saying that I'll need to review how Terms metadata is now being included in any of the exports that I'm interested in. And after this PR is merged, I plan on updating [the crosswalk](at https://docs.google.com/spreadsheets/d/10Luzti7svVTVKTA-px27oq3RxCUM-QbiTkm8iMd5C54), which people can reference to see how the most recent version of Dataverse is including Terms metadata in its exports.

[kcondon]

Seeing as this passes all functional tests and the only remaining issue is a very small but important doc change, we'll merge this and address the doc issue separately to move things along. thanks @janvanmansum for all the work!