added funderidtype and support for funder and sponsor roles to datacite metadata

[satra]

closes #168

• added funderidtype
• added support for funders and sponsors

according to @djarecka's test the current release mints a doi for the failing metadata schema on dandiarchive, so unsure what the exact issue was. nonetheless this update should be used moving forward since people use funder/sponsor synonymously in dandiset metadata.

[codecov]

Codecov Report

Patch coverage: 88.88% and no project coverage change.

Comparison is base (845cb17) 97.71% compared to head (0f5c77a) 97.72%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #167   +/-   ##
=======================================
  Coverage   97.71%   97.72%           
=======================================
  Files          17       17           
  Lines        1751     1758    +7     
=======================================
+ Hits         1711     1718    +7     
  Misses         40       40           

Flag	Coverage Δ
unittests	97.72% <88.88%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dandischema/tests/test_datacite.py	100.00% <ø> (ø)
dandischema/datacite.py	95.72% <88.88%> (+0.27%)	⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[djarecka]

lgtm

[yarikoptic]

according to @djarecka's test the current release mints a doi for the failing metadata schema on dandiarchive,

if we got to the bottom of it correctly during today's meeting, @djarecka didn't try to actually feed it to test datacite instance. Is the test where you added test case feeding it to it via https://github.com/dandi/dandi-schema/blob/HEAD/dandischema/tests/test_datacite.py#L17 ? would it fail if you remove the code you have added? without confirming the issue we would be not quite sure we're solving it fully.

[djarecka]

yes, looks like the error was not from to_datacite. During today meeting I was testing, but either I did something wrong, or I was getting inconsistent results... will update on slack

[djarecka]

I think my inconsistent results could be because clean_doi doesn't work as expected...

Any idea why I get *** requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: https://api.test.datacite.org/dois/10.80507/dandi.000333/0.0.0

[satra]

did you call this function:

dandi-schema/dandischema/tests/test_datacite.py

Line 17 in 4e35300

def datacite_post(datacite: dict, doi: str) -> None:

?

if so the doi has already been deleted because of the last step.

[djarecka]

but you can check that https://api.test.datacite.org/dois/10.80507/dandi.000333/0.0.0 exists

[satra]

i don't know what's going on, but have you tried running the other function so that it posts and deletes? that should tell you it's working or not.

[djarecka]

usually it works... but have no idea what is wrong with my small test, will debug a bit longer...

[djarecka]

it looks the method clean_doi doesn't work after running to_datacite with publish=True (additional attributes["event"] = "publish" is added). Do you understand why?

[satra]

i'm not sure i'm following, is there some script you can DM me to test?

[djarecka]

Just to update here:

• @satra explained to me that it is expected that if to_datacite is used with publish=True clean_doi will not delete the doi
• I've tried again running to_datacite and datacite_post on metadata from 000458 (after changing id, since I'm still not sure how to remove doi that I created last week) and it works. This is the exact code I run (in test_datacite:

dandi_id = metadata["identifier"]
dandi_id_noprefix = dandi_id.split(":")[1]
metadata.update(_basic_publishmeta(dandi_id=dandi_id_noprefix))
datacite = to_datacite(metadata)
datacite_post(datacite, metadata["doi"])

• I've tried this for https://api.test.datacite.org. If I have to check for a different one, let me know

[satra]

thanks @djarecka - did you try it with the specific metadata that failed on dandiarchive with the current released code (i.e. that there was an issue before) and this branch (now it works)?

[djarecka]

I used the link for 458 that you pointed me to on Friday.

Sorry, forgot to add that I don't have any issue with master, so still seems like I'm not recreating the initial problem

[satra]

@mvandenburgh - is there a way you can try to retrigger the doi generation on the server side for that dandiset that did not get a doi?

[mvandenburgh]

@mvandenburgh - is there a way you can try to retrigger the doi generation on the server side for that dandiset that did not get a doi?

Yes. Should I retrigger the doi generation with the code in this PR?

[satra]

@mvandenburgh - i would retest with what's on the server itself, since @djarecka can't replicate the issue with current main branch of this repo (without this PR).

[yarikoptic]

I wonder if worth

• make sure that the test record doesn't have funderidtype while funderid is there.
• if no error -- ask datacite folks why there is no error and show the error we received from a real server

[djarecka]

@mvandenburgh - could you please try to post the DOI with this branch? I've tested it in test https://api.test.datacite.org/dois with publish=True.

During the meeting on Monday I mentioned that api.test.datacite doesn't behave as api.datacite, but after checking carefully, that was my mistake, so hopefully this will solve the problem also with api.datacite

[djarecka]

@jwodder - do you know why dandi-cli are failing, not sure if the error comes from these changes...

[jwodder]

@djarecka See dandi/dandi-cli#1243 (comment).

[djarecka]

@mvandenburgh just tested and he was able to create the proper doi with this branch https://doi.datacite.org/dois/10.48324%2Fdandi.000458%2F0.230317.0039