Build integration with ROR identifiers

[mariapraetzellis]

As an organizational admin or super user creates a new organization in Roadmap this should ping the ROR API and record the associated identifier.

Ideally, we should also record the ROR for the Principle Investigator & Data Contact Person.

Outstanding question:

• For the DMPTool the organizational information is generated/recorded prior to authentication; for DMPOnline it is after. So, we'll need to determine corresponding workflows with this in mind.

[briri]

There are several areas where we can update the current text field/autocomplete selection boxes with this ROR lookup functionality.

• After a Shibboleth login (non-DMPTool workflows) and the user tells us which Org they belong to.
• On the Org details page allowing Org Admins and Super Admins to assign/correct ROR selections
• On the Project details page (as noted above) when the user is entering PI and Data Contact info
• On the Edit Profile page to allow a user to assign/change their Org affiliation

Other questions:

• We will run a script to attach ROR ids for existing Orgs in the database. Do we want to replace our current Org names with the official ROR names?
• We will have to allow the user to type in the name of a non-ROR Org (ROR only has research organizations ... no funder, etc.). Do we want to add the Org to the Orgs database table or continue assigning them to the is_other Org?

[briri]

Ok this turned out to be a lot more work than initially expected.

Database changes:

• Removed is_other Org and replaced with actual Org records
• Added a managed flag to Orgs (flag is now used everywhere we were previously checking the is_other flag
• Consolidated separate org_identifiers and user_identifiers table into a single identifiers table
• Added the ROR and Fundref identifier_schemes
• Added flags to help contextualize identifier schemes for_auth, for_orgs, for_plans and for_users
• Added a funder_id and org_id field to the Plans table

User Account Creation Screenshots:
New user account showing search results from ROR. The system will combine both local database results and ROR results (in situations where we have the same result for both, it shows the local database version ... e.g. 'University of California, Berkeley (UCB)' is shown instead of ROR's 'University of California, Berkeley (berkeley.edu)'

New user account creation showing the scenario where neither the local DB or ROR had a match. It warns the user but still allows them to continue:

User Functionality Screenshots:
Example of the new user's dashboard. It created the 'Example Test University' Org as an 'unmanaged' Org and attached the user to it.

Screenshot of the same Org selection box on the Edit Profile page. Allowing the user to select a local DB Org, one from the ROR registry or entering free text (this will also create an Org using the same logic the create user account uses):

Screenshot of the Create Plan page. This page continues to ONLY allow the user to select Orgs stored in the local database.

Screenshot of the Project Details page showing the new Funder selection box (This replaces the old funder_name field). If the template that was used to create the plan does not belong to the Funder the user specified on the create Plan page (e.g. user selected Funder 'ABC' which has no template so the default template was used) then the user will be allowed to change the Funder selection here:

Screenshot of the same page but for a scenario where the specified funder matches the creator of the template (e.g. Funder 'XYZ' was specified and the 'XYZ' template was used). The user is not able to change the Funder in this situation:

Super Admin Screenshots:
Screenshot of the Super Admin's Organizations page with a search for one of the new 'unmanaged' Orgs:

Screenshot of the Super Admin's view when editing an Org. Note the new 'Managed?' flag and 'Identifiers' section at the bottom. Also note that it was able to retrieve the URL from ROR.
The identifiers section allows an Org Admin to search the ROR registry and attach ROR and FUNDREF ids to the Org (note this action will NOT replace the name, abbreviation or url)

Screenshot of the Super Admin page to create a new Org. The 'Organization Lookup' box allows the admin to select one of the orgs from the ROR registry or enter free text if the org does not happen to be in the registry:

Org Admin Screenshots:
Screenshot of an Org Admin's view of their Org details. Note that in order for a user to be able to access the Admin menu (e.g. usage, users, etc.) their Org needs to be 'managed' and the user must have Org Admin privileges:

[mariapraetzellis]

Testing instructions from Brian:

account creation - Select an existing Org (e.g. UCOP)
account creation - Select a ROR Org (will have a URL domain)
account creation - Enter free text
edit the user profile and change Orgs (its the same control as account creation so no need to test all 3 scenarios)
Create a new plan. Select USGS or NSF and select a template
Create a new plan. tick the 'no funder' checkbox and then select a funder on the project details page
As a Super Admin - go to the Org Details page and change the managed flag
As a Super Admin - go to the Org Details page and lookup and Org in the Identifiers section and then save it (make sure the ROR and FUNDREF ids changed)
As a Super Admin - create a new Org by selecting a ROR entry
As a Super Admin - create a new Org but enter free text
As an Org Admin - make sure you do not see the managed flag but make sure you DO see your ROR and Fundref ids

Available on https://dmptool-stg.cdlib.org/

[mariapraetzellis]

When creating a plan there is a very long lag between the step to "Select the primary funding organization" and the Create Plan button being enabled. Is the time delay because it is calling to the ROR API? I'm not sure what else has changed that is causing the delay.

[briri]

No, there are no ROR calls on this page. It does make an ajax call back to the server to determine what templates are relevant to the user's selections.

The ajax call is triggered when the org/funder fields lose focus (or checkboxes get clicked). Maybe we need to change it to fire when the org/funder fields 'change'

[magdalenadrafiova]

Everything worked well but I was unable to do the superadmin functionalities. I checked my privileges and it said I am super admin - but I was getting error message saying that I am not authorised to replicate the steps

[briri]

Hi @magdalenadrafiova I updated your privileges for your Glasgow and Edinburgh uni accounts so that you have both Org Admin and Super Admin privileges (a super admin needs both). You should have access now.

[mariapraetzellis]

Sarah, Magdalena & I tested this together today. We are concerned with the number of duplicate orgs that will be created in this new process. When a user is about to create a new org we would like to modify the warning text to read:
"A new entry will be created for the organization you have named above. Please double check that your organization does not already appear in the list in a slightly different form."

Also, @xsrust can you deploy this to https://dmponline-test.dcc.ac.uk/ so Sarah/Magdalena can have some DMPOnline users test this out?

[sjDCC]

We've been doing testing on RoR and it looks good. We'll push it out to our users to look at too.

I noticed that the comment @mariapraetzellis made above to flag that a new org would be added to the database is not in the version we're testing