RFC: JSON-LD / Schema.org mapping

[mojodna]

As part of my work on STAC Browser, I've just merged preliminary JSON-LD support intended to facilitate indexing, searching, and display by Google Dataset Search.

I've tried to follow their guidelines, mapping Catalogs and Collections to schema.org DataCatalogs and Items to Datasets.

Catalog / Collection → DataCatalog

{
  "@context": "https://schema.org/",
  "@type": "DataCatalog",

  // required
  name: catalog.title,
  description: catalog.description, // as HTML

  // recommended
  identifier: catalog.properties["sci:doi"] || catalog.id,
  citation: catalog.properties["sci:citation"], // if available
  keywords: catalog.keywords,
  isBasedOn: catalog.url, // canonical STAC catalog URL (JSON)
  version: catalog.version,
  url: <STAC Browser URL>,
  // if available
  workExample: this.properties["sci:publications"].map(p => ({
     identifier: p.doi,
     citation: p.citation
   })),

  // if license is "proprietary"
  license: catalog.links.find(x => x.rel === "license").href,
  // if license is SPDX-compatible
  license: `https://spdx.org/licenses/${catalog.license}.html`,

  // if a spatial extent is available
  spatialCoverage = {
    "@type": "Place",
    geo: {
      "@type": "GeoShape",
      box: catalog.extent.spatial.join(" ")
    }
  },

  // if a temporal extent is available
  temporalCoverage: catalog.extent.temporal.map(x => x || "..").join("/"),

  // if a parent catalog is defined:
  isPartOf: {
    "@type": "DataCatalog",
    name: parent.title || parent.id, // if available
    isBasedOn: parent.url,
    url: <STAC Browser URL>
  },

  // for each child catalog:
  hasPart: {
    "@type": "DataCatalog",
    name: child.title,
    isBasedOn: child.url,
    url: <STAC Browser URL>
  },

  // for each referenced item:
  dataset: {
    identifier: item.id, // if available; requires loading the Item
    name: item.properties.title || item.id, // if available; requires loading the Item
    isBasedOn: item.url,
    url: <STAC Browser URL>
  }
}

providers are mapped according to roles (when multiple roles are specified, the provider is duplicated):

• licensor → copyrightHolder
• producer → producer
• processor → contributor
• host → provider

and rendered as:

{
  // ...
  [mapped role]: {
    description: provider.description, // if available
    name: provider.name,
     url: provider.url // if available
  }
}

Item → Dataset

{
  "@context": "https://schema.org/",
  "@type": "Dataset",

  // required
  name: item.properties.title || item.id,
  description: this.properties.description, // if available

  // recommended
  identifier: item.properties["sci:doi"] || item.id,
  citation: catalog.properties["sci:citation"], // if available
  keywords: collection.keywords || rootCatalog.keywords, // inherit collection / root catalog keywords, if available
  // if license is "proprietary"
  license: [item.links, collection.links, rootCatalog.links].find(x => x.rel === "license").href,
  // if license is SPDX-compatible
  license: `https://spdx.org/licenses/${item.properties["item:license"] || collection.license || rootCatalog.license}.html`,
  isBasedOn: item.url, // canonical STAC item URL (JSON)
  url: <STAC Browser URL>,
  // if available
  workExample: this.properties["sci:publications"].map(p => ({
     identifier: p.doi,
     citation: p.citation
   })),
  image: item.assets.thumbnail,

  // for associated collections + parent catalogs
  includedInDataCatalog: {
    isBasedOn: c.href,
    url: <STAC Browser URL>
  },

  spatialCoverage: {
    "@type": "Place",
    geo: {
      "@type": "GeoShape",
      box: item.bbox.join(" ")
    }
  },

  temporalCoverage: this.properties["dtr:start_datetime"]
    ? [
        this.properties["dtr:start_datetime"],
        this.properties["dtr:end_datetime"]
      ]
        .map(x => x || "..")
        .join("/")
    : item.properties.datetime,

  // for each asset in item.assets
  distribution: {
    contentUrl: asset.href,
    fileFormat: asset.type,
    name: asset.title
  }
};

This implementation is live (with pre-rendered HTML) at https://planet.stac.cloud. Hopefully in the coming days it will be better indexed by Google (I've submitted the sitemap), including by Dataset Search, at which point we can see how well this mapping does at being rendered.

Meanwhile, the OpenLink Structured Data Sniffer extension for Chrome will extract JSON-LD to allow inspection.

Thoughts?

Refs #285

[m-mohr]

Great work. Will dig deeper into it next week.

One quick note: If the license field is set to proprietary the url from the links should be taken.

[mojodna]

Good call, thanks. I've updated the mapping above as well as STAC Browser.

[m-mohr]

Great!

Another idea I had: The temporalCoverage for the Item could also be composed as an interval from the fields provided by the Datetime-Range-Extension (dtr).

The citation field from schema.org and publications field from our sci-extensions seem to map quite good for both Items and Catalogs.

[mojodna]

I've mapped sci:citation to citation and publications to workExample (my read is that the relationship is the inverse of citation).

[joshfix]

If we're interested in natively supporting json-ld in the STAC JSON output:

http://geojson.org/geojson-ld/

[m-mohr]

Is GeoJSON-LD compatible with what Google is asking for?

[joshfix]

Good question. Also, are they looking for JSON-LD only in HTML files?

[mojodna]

Is GeoJSON-LD compatible with what Google is asking for?

I don't think so. The docs refer to https://schema.org/GeoShape. It'd be nice if they supported both though (and maybe they do).

Also, are they looking for JSON-LD only in HTML files?

Yes, in the body of <script type="application/ld+json"> tags. I don't think they crawl JSON at the moment.

[cholmes]

Discussed on the call on April 22.
Status of this is that google doesn't actually use all the tags that it implies in their dataset search stuff. The consensus is just 'catalogs' should be exposed, as that's the level of 'search' they do there now.

It isn't clear exactly how to put this into the spec, as it is just one implementation for now.
Decision was made to keep this issue around, but to update the spec to talk at a more basic level about exposing one's data as HTML, etc. Likely in the catalog best practices document.