Validation schema and online editor

[simontaurus]

We are currently on the way implementing RO-Crate / ELN Fileformat for OpenSemanticLab.

On the way we will create a validation schema using OO-LD.

As discussed with @SteffenBrinckmann this issue to share the work early and provide a first preview (code):
playground example

[SteffenBrinckmann]

@simontaurus I like the idea: immediately show the content of the .eln file in a github.io and allow the user to inspect it.

[SteffenBrinckmann]

Hey,
I want to come back to the schema again, as it can help us clarify if something is complying. The ro-crate-metadata is a json and we should be able to create a schema for it. I have no experience in writing schema, but I am volunteering to create a github action to verify all .eln once the schema exists.

@FlorianRhiem, @nicobrandt, @NicolasCARPi : can you help; have you already created one?

There is no gui / preview aspect to this.

[NicolasCARPi]

TBH I'm surprised RO-Crate doesn't provide a schema. Seems it would be most helpful to everyone. This seems to discuss it: ResearchObject/ro-crate#33

Maybe the JSON-LD nature of it makes it hard to create... Given that all the nodes can accept a wide range of properties, which can themselves have values of different types and subtypes, I wonder if it's even possible to create a schema that will validate all eln. It will necessarily be restrictive/incomplete, and we will need to adjust it often, based on our use.

Here is a generated one from an elabftw metadata, as a starting point:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "@context": {
      "type": "string",
      "format": "uri"
    },
    "@graph": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "@id": {
            "type": "string"
          },
          "@type": {
            "type": "string"
          },
          "about": {
            "type": "object",
            "properties": {
              "@id": {
                "type": "string"
              }
            }
          },
          "conformsTo": {
            "type": "object",
            "properties": {
              "@id": {
                "type": "string",
                "format": "uri"
              }
            }
          },
          "dateCreated": {
            "type": "string",
            "format": "date-time"
          },
          "sdPublisher": {
            "type": "object",
            "properties": {
              "@id": {
                "type": "string"
              }
            }
          },
          "version": {
            "type": "string"
          },
          "author": {
            "type": "object",
            "properties": {
              "@id": {
                "type": "string"
              }
            }
          },
          "dateModified": {
            "type": "string",
            "format": "date-time"
          },
          "name": {
            "type": "string"
          },
          "encodingFormat": {
            "type": "string"
          },
          "url": {
            "type": "string",
            "format": "uri"
          },
          "genre": {
            "type": "string"
          },
          "creativeWorkStatus": {
            "type": "string"
          },
          "identifier": {
            "type": "string"
          },
          "keywords": {
            "type": "string"
          },
          "step": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "@type": {
                  "type": "string"
                },
                "position": {
                  "type": "integer"
                },
                "creativeWorkStatus": {
                  "type": "string"
                },
                "itemListElement": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "@type": {
                        "type": "string"
                      },
                      "text": {
                        "type": "string"
                      }
                    }
                  }
                }
              }
            }
          },
          "hasPart": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "@id": {
                  "type": "string"
                }
              }
            }
          },
          "comment": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "@id": {
                  "type": "string"
                },
                "@type": {
                  "type": "string"
                },
                "dateCreated": {
                  "type": "string",
                  "format": "date-time"
                },
                "text": {
                  "type": "string"
                },
                "author": {
                  "type": "object",
                  "properties": {
                    "@id": {
                      "type": "string"
                    }
                  }
                }
              }
            }
          }
        },
        "required": ["@id", "@type"]
      }
    }
  },
  "required": ["@context", "@graph"]
}

[FlorianRhiem]

@FlorianRhiem, @nicobrandt, @NicolasCARPi : can you help; have you already created one?

I haven't created one, rather I'm using a custom implemented parser (after using the built-in json module for parsing the data as json in the first place). This way I can directly split things up in ways useful for SampleDB, create warnings or error messages based upon that, etc. It might be better to have a unified parser, but doing it this way got things rolling quickly.

[simontaurus]

The main issue is that RO-CRATE specifies in principle an RDF graph, serialized as flattened and compacted JSON-LD (@graph with a list of nodes). This makes any syntactical validation (like JSON-SCHEMA) limited, in comparison to semantic RDF-"SCHEMAS" like SHACL (which, vise verse, have their limits in adaption and tool availability)

However, looking more closely, the RO-CRATE spec is not pure semantically but also syntactically:
e.g.

@type: MUST be Dataset
datePublished: MUST be a string in ISO 8601 date format and SHOULD be specified to at least the precision of a day, MAY be a timestamp down to the millisecond.

Pure semantically would mean, e.g. "the triple <./.> <rdf:type> <http://schema.org/Dataset> is present".
This leads to the interpretation (and implementations, also in https://github.com/TheELNConsortium/TheELNFileFormat/blob/master/tests/test_01_params_metadata_json.py#L53) that the named keys (e.g. @type) must be present but in fact this two JSON-LD files are semantically equivalent:

{
  "@context": "https://w3id.org/ro/crate/1.1/context",
  "@type": "Dataset"
}

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "type": "@type"
    }
  ],
  "type": "Dataset"
}

In addition, RO-Crate inherits the very lax handling of data type, cardinality, range and required-ness of properties further comlicating validation and consumer implementation.
Therefore we would suggest to create at least for the ELN format syntactically (but sematically annotated) schemas (see OO-LD) for a normalized tree-shape of the ro-create-metadata.json. Luckily JSON-LD can help use here with simply applying the framing algorithm, in combination with some aliases (@id => id, @type => type) in oder to get valid variable names e.g. in python.

Early ELN-Fileformat Schema Draft

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ],
  "title": "RO-Crate",
  "type": "object",
  "version": "0.1.1",
  "required": [
    "id",
    "type",
    "conformsTo",
    "sdPublisher"
  ],
  "definitions": {
    "Thing": {
      "type": "object",
      "required": [
        "type"
      ],
      "properties": {
        "type": {
          "type": "string",
          "default": "Thing"
        },
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "description": {
          "type": "string"
        }
      }
    },
    "Organization": {
      "allOf": [
        {
          "$ref": "#/definitions/Thing"
        }
      ],
      "type": "object",
      "required": [
        "type"
      ],
      "properties": {
        "type": {
          "type": "string",
          "default": "Organization"
        },
        "url": {
          "type": "string",
          "format": "url"
        },
        "areaServed": {
          "type": "string"
        },
        "slogan": {
          "type": "string"
        },
        "logo": {
          "type": "string",
          "format": "url",
          "links": [
            {
              "href": "{{self}}",
              "type": "img/png"
            }
          ]
        },
        "parentOrganization": {
          "$ref": "#/definitions/Organization"
        }
      },
      "options": {
        "display_required_only": true
      }
    },
    "Person": {
      "allOf": [
        {
          "$ref": "#/definitions/Thing"
        }
      ],
      "title": "Person",
      "type": "object",
      "properties": {
        "email": {
          "type": "string",
          "format": "email"
        },
        "familyName": {
          "type": "string"
        },
        "givenName": {
          "type": "string"
        }
      }
    },
    "CreativeWork": {
      "allOf": [
        {
          "$ref": "#/definitions/Thing"
        }
      ],
      "title": "CreativeWork",
      "type": "object",
      "properties": {
    "dateCreated": {
      "type": "string",
      "format": "datetime-local",
      "options": {
        "flatpickr": {}
      }
    },
    "dateModified": {
      "type": "string",
      "format": "datetime-local",
      "options": {
        "flatpickr": {}
      }
    },
        "keywords": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      }
    },
    "File": {
      "allOf": [
        {
          "$ref": "#/definitions/CreativeWork"
        }
      ],
      "title": "File",
      "type": "object",
      "id": "file",
      "required": [
        "type",
        "id"
      ],
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "File"
          ]
        },
        "_id": {
          "type": "string"
        },
        "encodingFormat": {
          "type": "string"
        },
        "id": {
          "type": "string",
          "format": "url",
          "options": {
            "upload": {
              "upload_handler": "testUploadHandler"
            }
          },
          "links": [
            {
              "href": "dummy",
              "rel": "view / download"
            }
          ]
        }
      },
      "not": { "required": [ "hasPart" ] },
      "options": {
        "display_required_only": true
      }
    },
    "Dataset": {
      "allOf": [
        {
          "$ref": "#/definitions/CreativeWork"
        }
      ],
      "title": "Dataset",
      "type": "object",
      "required": [
        "type",
        "id"
      ],
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "Dataset"
          ]
        },
        "about": {
          "$ref": "#/definitions/Thing"
        },
        "hasPart": {
          "type": "array",
          "format": "tabs",
          "items": {
        "discriminator": {
        "propertyName": "type",
        "mapping": {
          "File": "#/definitions/File",
          "Dataset": "#/definitions/Dataset"
        }
      },
            "oneOf": [
              {
                "$ref": "#/definitions/File"
              },
              {
                "$ref": "#/definitions/Dataset"
              }
            ]
          }
        }
      },
      "options": {
        "display_required_only": true
      }
    }
  },
  "properties": {
    "id": {
      "type": "string",
      "enum": [
        "ro-crate-metadata.json"
      ]
    },
    "type": {
      "type": "string",
      "enum": [
        "CreativeWork"
      ]
    },
    "conformsTo": {
      "type": "string",
      "enum": [
        "https://w3id.org/ro/crate/1.1"
      ]
    },
    "version": {
      "type": "string"
    },
    "sdPublisher": {
      "$ref": "#/definitions/Organization"
    },
    "about": {
      "$ref": "#/definitions/Dataset",
      "properties": {
        "id": {
          "enum": [
            "./"
          ]
        }
      },
      "default": {
        "hasPart": []
      }
    }
  },
  "options": {
    "_display_required_only": true
  }
}

This would provide use three outcomes:

1. programming language agnostic validation by using JSON-LD normalization + JSON-SCHEMA
2. code generation to automate the parsing and serialization
3. UI generation for generic viewers / editors

As a demo for 1. and 3.,
ELN Fileformat Playground - PASTA.eln
reveals that Steffen has nested a Dataset in a File, which could at least be discussed.

For 2., throwing the same schema at OO-LD Python playground gives us pydantic dataclasses both for validation and implementation.

Generated Dataclasses

from __future__ import annotations

from enum import Enum
from typing import List, Literal, Optional, Union

from pydantic import BaseModel, EmailStr, Field


class Id(Enum):
    ro_crate_metadata_json = "ro-crate-metadata.json"


class Type(Enum):
    CreativeWork = "CreativeWork"


class ConformsTo(Enum):
    https___w3id_org_ro_crate_1_1 = "https://w3id.org/ro/crate/1.1"


class Thing(BaseModel):
    type: str
    id: Optional[str] = None
    name: Optional[str] = None
    description: Optional[str] = None


class Organization(Thing):
    type: str
    url: Optional[str] = None
    areaServed: Optional[str] = None
    slogan: Optional[str] = None
    logo: Optional[str] = Field(None, links=[{"href": "{{self}}", "type": "img/png"}])
    parentOrganization: Optional[Organization] = None


class Person(Thing):
    email: Optional[EmailStr] = None
    familyName: Optional[str] = None
    givenName: Optional[str] = None


class CreativeWork(Thing):
    dateCreated: Optional[str] = Field(None, options={"flatpickr": {}})
    dateModified: Optional[str] = Field(None, options={"flatpickr": {}})
    keywords: Optional[List[str]] = None


class Type1(Enum):
    File = "File"


class File(CreativeWork):
    type: Literal["File"]
    field_id: Optional[str] = Field(None, alias="_id")
    encodingFormat: Optional[str] = None
    id: str = Field(
        ...,
        links=[{"href": "dummy", "rel": "view / download"}],
        options={"upload": {"upload_handler": "testUploadHandler"}},
    )


class Type2(Enum):
    Dataset = "Dataset"


class ROCrate(BaseModel):
    id: Id
    type: Type
    conformsTo: ConformsTo
    version: Optional[str] = None
    sdPublisher: Organization
    about: Optional[Dataset] = Field(
        default_factory=lambda: Dataset.parse_obj({"hasPart": []})
    )


class Dataset(CreativeWork):
    type: Literal["Dataset"]
    about: Optional[Thing] = None
    hasPart: Optional[List[Union[File, Dataset]]] = Field(None, discriminator="type")
    id: str

While this approach never forbidds additional properties we can easily define in a machine readable way which properties we expect to be used and how we expect them to be used. Also we can define subclasses of Dataset or File that have a strict definition of metadata in order to allow automated processing, e.g. CsvFile with CSVW conform column annotations.

What do you think?

[SteffenBrinckmann]

@simontaurus How do we continue here, this issue seems stalled.

[simontaurus]

#116 and #98 closed makes it easier to continue here - I will work on it

[ptsefton]

How did the framing efforts go? I tried this in the early development on RO-Crate and I could never get it to work relliably -- even setting aside issues with cycles in the graph.

[simontaurus]

Using JSON-LDs framing is little bit tricky but definitely helps to process RO-Crates. Does the following address the issues you have encountered?

Lets take this example RO-Crate JSON-LD with

• a primary Dataset tree
• an additional author node referenced twice
• a circular backref from the author node to the tree

Example RO-Crate JSON-LD

{ "@context": "https://w3id.org/ro/crate/1.1/context",
  "@graph": [

    {
      "@type": "CreativeWork",
      "@id": "ro-crate-metadata.json",
      "conformsTo": {"@id": "https://w3id.org/ro/crate/1.1"},
      "about": {"@id": "./"},
      "description": "RO-Crate Metadata File Descriptor (this file)"
    },
    {
      "@id": "./",
      "@type": "Dataset",
      "name": "Example RO-Crate",
      "description": "The RO-Crate Root Data Entity",
      "hasPart": [
        {"@id": "data1.txt"},
        {"@id": "data2.txt"}
      ]
    },
    {
      "@id": "data1.txt",
      "@type": "File",
      "description": "One of hopefully many Data Entities",
      "author": {"@id": "#alice"},
      "contentLocation":  {"@id": "http://sws.geonames.org/8152662/"}
    },
    {
      "@id": "data2.txt",
      "@type": "File",
      "author": {"@id": "#alice"}
    },
    {
      "@id": "#alice",
      "@type": "Person",
      "name": "Alice",
      "description": "One of hopefully many Contextual Entities",
      "knowsAbout": [{"@id": "data1.txt"}, {"@id": "data1.txt"}]
    },
    {
      "@id": "http://sws.geonames.org/8152662/",
      "@type": "Place",
      "name": "Catalina Park"
    }
 ]
}

Using an empty frame

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ]
}

leads to multiple trees, starting from every node in the @graph.

Forcing @type": "CreativeWork" on the root node limits the result to a main tree

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ],
  "@type": "CreativeWork"
}

Hower, due to default "@embed": "@once", #alice is still refered by its @id on the second occurence.
To overcome this we can set "@embed": "@always" globally.

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ],
  "@type": "CreativeWork",
  "@embed": "@always"
}

Now we have a single tree with all leaves embedded (see also JSON-LD Playground) so we can use a (JSON-)SCHEMA to validate the schema:Person objects ro:author points to. Circular backrefs through knowsAbout are stopped by the framing algorithm after one interation.

Single tree, all leaves embedded

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ],
  "id": "ro-crate-metadata.json",
  "type": "CreativeWork",
  "conformsTo": {
    "id": "https://w3id.org/ro/crate/1.1"
  },
  "about": {
    "id": "./",
    "type": "Dataset",
    "description": "The RO-Crate Root Data Entity",
    "hasPart": [
      {
        "id": "data1.txt",
        "type": "File",
        "author": {
          "id": "#alice",
          "type": "Person",
          "description": "One of hopefully many Contextual Entities",
          "knowsAbout": {
            "id": "data1.txt"
          },
          "name": "Alice"
        },
        "contentLocation": {
          "id": "http://sws.geonames.org/8152662/",
          "type": "Place",
          "name": "Catalina Park"
        },
        "description": "One of hopefully many Data Entities"
      },
      {
        "id": "data2.txt",
        "type": "File",
        "author": {
          "id": "#alice",
          "type": "Person",
          "description": "One of hopefully many Contextual Entities",
          "knowsAbout": {
            "id": "data1.txt",
            "type": "File",
            "author": {
              "id": "#alice"
            },
            "contentLocation": {
              "id": "http://sws.geonames.org/8152662/",
              "type": "Place",
              "name": "Catalina Park"
            },
            "description": "One of hopefully many Data Entities"
          },
          "name": "Alice"
        }
      }
    ],
    "name": "Example RO-Crate"
  },
  "description": "RO-Crate Metadata File Descriptor (this file)"
}

To stop embedding at a defined edge we can use an extended frame to keep ro:author values as separate nodes (we need to know the maximum nesting level here)

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
  	}
  ],
  "@type": "CreativeWork",
  "@embed": "@always",
  "about": {
    "@embed": "@always",
    "hasPart": {
      "@embed": "@always",
      "author": {
        "@embed": "@never"
      }
    }
  }
}

JSON-SCHEMA as it is can validate the value of ro:author now only as a "type": "string" but we came up with a extended notations that allows the iterative schema validation on references by defining "range": {"allOf": {"$ref": "#defs/Person"}} (see OO-LD)
The union graph with a frame selecting only "@type": "schema:Person" would be the base for this.

Union Graph with separate author obects

{
  "@context": [
    "https://w3id.org/ro/crate/1.1/context",
    {
      "id": "@id",
      "type": "@type"
    }
  ],
  "@graph": [
  {
  "id": "ro-crate-metadata.json",
  "type": "CreativeWork",
  "conformsTo": {
    "id": "https://w3id.org/ro/crate/1.1"
  },
  "about": {
    "id": "./",
    "type": "Dataset",
    "description": "The RO-Crate Root Data Entity",
    "hasPart": [
      {
        "id": "data1.txt",
        "type": "File",
        "author": {
          "id": "#alice"
        },
        "contentLocation": {
          "id": "http://sws.geonames.org/8152662/",
          "type": "Place",
          "name": "Catalina Park"
        },
        "description": "One of hopefully many Data Entities"
      },
      {
        "id": "data2.txt",
        "type": "File",
        "author": {
          "id": "#alice"
        }
      }
    ],
    "name": "Example RO-Crate"
  },
  "description": "RO-Crate Metadata File Descriptor (this file)"
  },
  {
  "id": "#alice",
  "type": "Person",
  "description": "One of hopefully many Contextual Entities",
  "knowsAbout": {
    "id": "data1.txt"
  },
  "name": "Alice"
  }
}

SHACL shapes would be another approach without this limitation but beeing more distant to object oriented programming.

[SteffenBrinckmann]

Is this the approach that allotrope takes? They should have the same challenge. Here an example link for fast reference: https://gitlab.com/allotrope-public/asm-jupyter-demo/-/blob/main/ASM_Juypter_Notebook_Demo.ipynb?ref_type=heads

[NicolasCARPi]

@simontaurus think we can close this issue? Validator: https://check-eln.streamlit.app/

[CAOShurong]

I tested the checker against current reference archives before deciding whether the validator is enough to close this issue.

Scope: the four check functions in checkELN at 2bdff1b, compared with the same four functions in TheELNFileFormat at 19028dd. I ran them headlessly with the checker's pinned dependencies against all 11 .eln archives currently under examples/; I did not automate the Streamlit UI.

Two reproducible gaps look relevant:

1. The checker copy has drifted from the current conformance checks. For the current SciLog example (SHA-256 2a264b20683f3c30aeb10ef7e4223fe8d7a141505e41855ddfc52244e90f43a0), checkELN returned pass / fail / pass / fail for PyPI RO-Crate / validator / params / schema. The current TheELN checks returned pass / pass / pass / pass for the same file in the same environment. The schema failures are the array forms of @context and @type: the checker schema accepts only strings, while the current consortium schema accepts strings or arrays. The validator failure was ro-crate-1.1_5.3; the current check has the RO-Crate 1.2 compatibility handling for SciLog and SampleDB, which the checker copy lacks.

2. Neither implementation checks the mandatory outer archive layout. Starting from the current OpenSemanticLab minimal example (SHA-256 efe2ac320d754f81d75691c7325d192b3f0300043200ffe09491c36760ae739c), I appended one root-level entry named unexpected-root.txt. The resulting archive has two top-level entries, MinimalExample and unexpected-root.txt, so it violates the specification's single-root-folder MUST. All four checks returned success in both implementations. The derived archive SHA-256 is e9f329deb4e592c1b2ed9e6b7215d953468423eaf4264f28280184c749efbcc8; it can be reproduced with:

from pathlib import Path
from shutil import copyfile
from zipfile import ZipFile

src = Path("examples/OpenSemanticLab/MinimalExample.osl.eln")
dst = Path("opensemanticlab-extra-root-entry.eln")
copyfile(src, dst)
with ZipFile(dst, "a") as archive:
    archive.writestr("unexpected-root.txt", "x")

The online checker is useful, but these false-negative and false-positive cases mean I would not treat it as a complete .eln conformance gate yet. Keeping this issue open until the app consumes the current shared checks/schema and validates the outer ZIP layout would give it a concrete, testable completion criterion.