add testing that a prototypical dandiset metadata validates using converted to json schema

[yarikoptic]

tools/pubschemata.py publishes schema as json schema to https://github.com/dandi/schema . Conversion to json might "mutate" it so a correct according to pydantic's model metadata would not validate using that json schema and jsonschema python package (is that what you were using @dchiquit? found no jsonschema import in dandi-api though).

• move publish_model_schemata from tools/pubschemata.py into dandi/metadata.py
• add a function to validate metadata given provided json schema directory (as produced by publish_model_schemata)
• verify that metadata validates using that schema and jsonschema:
  • add tests: in http://github.com/dandi/dandi-cli/blob/HEAD/dandi/tests/test_metadata.py we already have some quite detailed sample cases for dandiset level metadata migration.

[jwodder]

add a function to validate metadata given provided json schema directory

What format should this function take the metadata in? Deserialized JSON?

[yarikoptic]

yes

[dchiquito]

dandi-api does not do any schema validation yet. @AlmightyYakob has been working on the metadata editor which lives in dandiarchive, but I have no idea what schema he has been using.

[jjnesbitt]

dandi-api does not do any schema validation yet. @AlmightyYakob has been working on the metadata editor which lives in dandiarchive, but I have no idea what schema he has been using.

I'm using this version https://github.com/dandi/schema/blob/master/releases/1.0.0-rc1/dandiset.json

[jwodder]

@yarikoptic The asset and metadata samples in test_metadata.py do not validate against the JSON Schema; they appear to have various fields set to None that shouldn't be None. What should be done about this?

[yarikoptic]

could you please give some example(s)? most likely then they should be announced Optional and thus allow to be None, or am I misunderstanding?

[jwodder]

@yarikoptic Example, from trying to validate the object in test_dandimeta_migration:

E   jsonschema.exceptions.ValidationError: None is not of type 'array'
E
E   Failed validating 'type' in schema['properties']['studyTarget']:
E       {'description': 'What the study is related to',
E        'items': {'anyOf': [{'format': 'uri',
E                             'maxLength': 65536,
E                             'minLength': 1,
E                             'type': 'string'},
E                            {'type': 'string'}]},
E        'nskey': 'dandi',
E        'title': 'Study Target',
E        'type': 'array'}
E
E   On instance['studyTarget']:
E       None

[yarikoptic]

aha -- so here it is None vs array. While serializing, use meta.json(exclude_unset=True, exclude_none=True) - should help

[jwodder]

@yarikoptic With that change, I instead get the error "'citation' is a required property". Apparently, a required field is missing.

[yarikoptic]

hm, I thought we would be failing at pydantic level... strange. To move forward, let's make any such optional as well but with obvious annotation, e.g. via

diff --git a/dandi/models.py b/dandi/models.py
index c08bc39..7ecae73 100644
--- a/dandi/models.py
+++ b/dandi/models.py
@@ -16,6 +16,7 @@ from .model_types import (
 
 from .utils import name2title
 
+TempOptional = Optional
 
 def create_enum(data):
     """Convert a JSON-LD enumeration to an Enum"""
@@ -592,7 +593,7 @@ class DandiMeta(CommonModel):
         min_items=1,
     )
 
-    citation: str = Field(readOnly=True, nskey="schema")
+    citation: TempOptional[str] = Field(readOnly=True, nskey="schema")
 
     # From assets
     assetsSummary: AssetsSummary = Field(readOnly=True, nskey="dandi")

[jwodder]

@yarikoptic I've now run into this error, and I'm not sure what part should be modified to address it:

E   jsonschema.exceptions.ValidationError: 'CC-BY-4.0' is not of type 'array'
E       
E   Failed validating 'type' in schema['properties']['license']:
E       {'description': 'License of item.',
E        'items': {'$ref': '#/definitions/LicenseType'},
E        'nskey': 'schema',
E        'title': 'License',
E        'type': 'array'}
E
E   On instance['license']:
E       'CC-BY-4.0'

[satra]

@jwodder - feel free to make up a fake dandiset that validates. in this instance the input should be a list i believe.

[jwodder]

@satra Shouldn't the Dandiset that's already used in the tests be the one that should validate? Is the problem that the input to migrate2newschema() is wrong, that migrate2newschema() doesn't convert its input correctly, or that the field type is wrong?

[satra]

the current tests all use unvalidated. for them to validate, we would need to fix the inputs and/or the conversion piece. would probably be good to move towards a valid, even if made up, input.