Skip to content

Commit

Permalink
Merge branch 'rosiel-documenting-rdf-generation'
Browse files Browse the repository at this point in the history
  • Loading branch information
@dannylamb
dannylamb committed Nov 12, 2019
2 parents 6941603 + 09df8d4 commit 2da9dbf
Show file tree
Hide file tree
Showing 2 changed files with 211 additions and 31 deletions.
240 changes: 209 additions & 31 deletions docs/islandora/rdf-mapping.md
View file
Original file line number Diff line number Diff line change
@@ -1,39 +1,134 @@
## RDF Mapping
RDF mapping is aligning drupal fields to RDF ontology properties. For example the `title` field of a content model can be mapped to `dcterms:title` and/or `schema:title`. In Islandora 8, triples expressed by these mappings get synced to Fedora and indexed in the Blazegraph triplestore.
# RDF Generation

Drupal provides default RDF mappings for its core entities. For instance, the article content type's RDF mapping is specified in the `rdf.mapping.node.page.yml` config. One can export this RDF mapping by going to Configuration Synchronization: `http://localhost:8000/admin/config/development/configuration/single/export`, selecting `RDF` for Configuration type and choosing `node.article`.
## Summary
In Islandora, the **JSON-LD Module** transforms nodes (or media, or taxonomy terms) into the RDF that is synced into Fedora and the Triplestore. It uses RDF mappings, a concept defined by the **RDF Module**, and exposes them through the **REST API** at `?_format=jsonld`.

RDF mappings are defined/stored in Drupal as a YAML file. Currently, Drupal 8 does not have a UI to create/update RDF mappings to ontologies other than Schema.org. There is a project underway to develop a [UI](https://github.com/Islandora-CLAW/CLAW/issues/647) to support RDF mappings to any ontology. However, until then exporting/importing RDF yml files via Configuration Synchronization is the primary method to create/update RDF mappings.
## Background

### Structure of RDF YAML file
Below is an example of RDF mapping. It is the current version of the RDF mapping of Islandora collection content model (`rdf.mapping.node_type.collection.yml`). `types` specify the `rdf:type` of the resource or content model. `fieldMappings` specify all fields of that bundle and their RDF property mappings. One field can be mapped to more than one RDF property. It is a simple flat list.
A quick overview of JSON-LD, the RDF module, and the REST API.

types and fieldMappings
### The JSON-LD Syntax
[JSON-LD](https://www.w3.org/2013/dwbp/wiki/RDF_AND_JSON-LD_UseCases) is a syntax which can be used to express RDF (like Turtle, or RDF XML), that is written in JSON, because devs like JSON and it's web-friendly. The JSON-LD syntax was designed for including Linked Data within HTML of web pages (similar to microdata or RDFa). Instead of nesting the RDF predicates within _existing_ HTML tags as RDFa does, JSON-LD lets you put a solid blob of Linked Data inside a `<script>` tag. JSON-LD can also function as a standalone document, which is how we're using it.

### RDF (Drupal Module)
The **RDF Module** is part of Drupal Core, but has no official documentation. The RDF Module embeds RDFa, a form of linked data, within the Drupal-generated HTML when you load the web page for a node, media, or taxonomy term. Official line is that this will allow Google to provide "rich snippets" such as star-ratings, contact info, and business hours. As an example of Drupal-provided RDFa:

```html
<h1 class="page-header">
<span property="schema:title">My cat</span>
</h1>
```
The `property="schema:title"` is markup generated by Drupal's RDF module that identifies the value "My cat" as the schema.org `title` of this page. A node's fields (such as `field_tags`) and properties (such as `author`) can be mapped to RDF according to a bundle-specific "mapping" that is stored within Drupal. In Drupal8-ese, RDF mappings are configuration entities. Drupal doesn't have a good UI for editing RDF mappings, but you can create, read, and update them as YAML files using Drupal's Configuration Synchronization interface (see section below on How to Edit an RDF Mapping)..


### REST API
The pattern of using `?_format=` to get a different representation of content is provided by the RESTful Web Services (rest) module. It allows other services to interact with Drupal entities through HTTP requests (`GET`, `POST`, `PATCH`, and `DELETE`). Which operations are allowed, and with what formats (such as `xml`, `json`, and `jsonld`) is configured at `admin/config/services/rest/`. Note that only `jsonld` uses RDF mappings; the `json` and `xml` formats expose a structured object based on how Drupal sees the entity. Access to these alternate formats throught the REST API corresponds to permissions on the entity, so anyone with `access content` permission can view the JSON-LD version of that content. This is new as of[ Drupal 8.2](https://www.drupal.org/docs/8/api/restful-web-services-api/restful-web-services-api-overview#practical).

For more information on interacting with Drupal entities via REST requests, see [An Introduction to RESTful Web Services in Drupal 8](https://drupalize.me/blog/201401/introduction-restful-web-services-drupal-8).

## JSON-LD Module

Using the RDF mapping configurations provided by the RDF module, the JSON-LD Module exposes the RDF-mapped entity in JSON-LD, through the REST API, at `node/[nid]?_format=jsonld` (for nodes; for media and terms, at `media/[mid]?_format=jsonld` and `taxonomy/term/[tid]?_format=jsonld`).

- The JSON-LD module will only work with mappings that include a value under `types` (which maps to `rdf:type` - see below, under Structure of an RDF Mapping).
- The JSON-LD module provides a hook so other modules can alter the entity before it gets mapped. The `islandora` module uses this hook to trigger any "Map URI to Predicate" and "Alter JSON-LD Type" reactions that are configured in Contexts. `islandora_defaults` provides the two Contexts - "All Media" and "Content" - that configure these to occur on Media and Repository Item nodes.
- The JSON-LD module adds RDF datatypes to the RDF values, and includes a mapping of Drupal field types to RDF datatypes.
- The JSON-LD module provides a hook to alter its Drupal field type to RDF datatype mapping.
- The JSON-LD module has a configuration option that can cause the `?_format=jsonld` to be part of, or not part of, the URIs of Drupal objects. On an out-of-the-box claw-playbook, this string is stripped, but by default on a fresh install of the jsonld module, it is not.


## RDF Mappings
In an out-of-the-box claw-playbook, the RDF mappings that exist were loaded from config files, and correspond to the rdf.mapping.[...].yml files located in:

- `[drupal modules directory]/islandora/modules/islandora_core_feature/config/install/` (media and taxonomy terms)
- `[drupal modules directory]/islandora_defaults/config/install/` (`repository_item` and the `islandora_access` vocabulary)
- `[drupal modules directory]/controlled_access_terms/modules/controlled_access_terms_defaults/config/install/` (the default `corporate_body`, `family`, `geo_location`, `person`, `resource_type` and `subject` vocabularies)
- `[drupal web root]/core/profiles/standard/config/install/` (articles, pages, comments, and tags).

Once loaded by modules, configuration .yml files are not live so **editing them will not change the existing configuration**. However, for modules that are Features, it is possible to re-import the changed configuration files at `admin/config/development/features` (todo: link to further reading on Features).

### How to edit an RDF Mapping

Once loaded, RDF mappings can be customized for the needs of a particular site through Drupal's Configuration Synchronization UI at `admin/config/development/configuration`. They can be exported, modified, and re-imported one-at-a-time by choosing the "Single Item" option on the Export/Import tabs. You can also create new RDF mappings (e.g. for a custom content type) and load them through this interface, by copying an existing mapping and changing the appropriate values.

!!! note "Contributed module for RDF Mappings"
A custom module `rdfui` exists, and is installed-but-not-enabled on boxes provisioned by the claw-playbook. We don't use it because it is very rudimentary and limited to the schema.org vocabulary. We have an [open ticket](https://github.com/Islandora-CLAW/CLAW/issues/647) to develop a UI to support RDF mappings to any ontology. Contributions welcome.

- A number of namespaces such as `ldp`, `ebucore`, `pcdm`, are `premis` are registered in `islandora.module` using `hook_rdf_namespaces()`. To register your own namespaces, you will need to create a custom module that implements that hook.
- If you import a configuration that uses a namespace that is not registered, bad things will happen silently.


### Structure of an RDF Mapping
Below is an example of an RDF mapping as a .yml (YAML) file. It is the current version of the RDF mapping of the Repository Item (`islandora_object`) bundle, provided by `islandora_defaults` and exportable as `rdf.mapping.node.islandora_object.yml`).

- The top level key `types` specifies the `rdf:type` of the resource or content model. `field_model`, a required field of Islandora objects, also gets mapped to `rdf:type` through an arcane back-end process.
- The top level key `fieldMappings` specifies fields attached to that bundle and their RDF property mappings. One field can be mapped to more than one RDF property. It is a simple flat list.
- `datatype_callback` : [needs documentation]
- `mapping_type: rel` : [needs documentation]

## Sample RDF Mapping
```yml
langcode: en
status: true
dependencies:
config:
- node.type.islandora_collection
module:
- islandora
- node.type.islandora_object
enforced:
module:
- islandora_collection
id: node.islandora_collection
- islandora_defaults
module:
- node
id: node.islandora_object
targetEntityType: node
bundle: islandora_collection
bundle: islandora_object
types:
- 'pcdm:Collection'
- 'schema:CollectionPage'
- 'pcdm:Object'
fieldMappings:
field_alternative_title:
properties:
- 'dc:alternative'
field_edtf_date:
properties:
- 'dc:date'
datatype_callback:
callable: 'Drupal\controlled_access_terms\EDTFConverter::dateIso8601Value'
field_edtf_date_created:
properties:
- 'dc:created'
datatype_callback:
callable: 'Drupal\controlled_access_terms\EDTFConverter::dateIso8601Value'
field_edtf_date_issued:
properties:
- 'dc:issued'
datatype_callback:
callable: 'Drupal\controlled_access_terms\EDTFConverter::dateIso8601Value'
field_description:
properties:
- 'dc:description'
field_memberof:
field_extent:
properties:
- 'dc:extent'
field_identifier:
properties:
- 'dc:identifier'
field_member_of:
properties:
- 'pcdm:memberOf'
mapping_type: rel
field_resource_type:
properties:
- 'dc:type'
mapping_type: rel
field_rights:
properties:
- 'dc:rights'
field_subject:
properties:
- 'dc:subject'
mapping_type: rel
field_weight:
properties:
- 'co:index'
title:
properties:
- 'dc:title'
Expand All @@ -53,18 +148,101 @@ fieldMappings:
mapping_type: rel
```
### Viewing RDF mapping of a resource
Please see the following tutorial to configure and view resources via REST request:
[An Introduction to RESTful Web Services in Drupal 8](https://drupalize.me/blog/201401/introduction-restful-web-services-drupal-8). To get the JsonLD mapping, use jsonld format. Example request URL: `http://localhost:8000/node/1?_format=jsonld`

### Altering an existing RDF mapping
Each Islandora content models come with a RDF mapping. It can be exported similar to above article's rdf mapping, modified and imported back by going to here: `http://localhost:8000/admin/config/development/configuration/single/import`.

### Defining a new RDF mapping
A RDF mapping need to be created for a new or custom content model/type. The following steps describe the procedure:
* Export RDF mapping of a similar content type
* Export form display yml to get the list of the fields (ex `core.entity_form_display.node.your_content_type.default.yml`)
* Remove any cache and UUID related elements
* Add/modify the fields and ontology properties as needed, following the same syntax
* Name it following the convention (ex `rdf.mapping.node_type.your_content_type.yml`)
* Import it by going to `http://localhost:8000/admin/config/development/configuration/single/import`
## Sample JSON-LD
```json
{
"@graph":[
{
"@id":"http:\/\/future.islandora.ca\/node\/8",
"@type":[
"http:\/\/pcdm.org\/models#Object"
],
"http:\/\/purl.org\/dc\/terms\/title":[
{
"@value":"lasmomias de uninpahu",
"@language":"fa"
}
],
"http:\/\/schema.org\/author":[
{
"@id":"http:\/\/future.islandora.ca\/en\/user\/1"
}
],
"http:\/\/schema.org\/dateCreated":[
{
"@value":"2019-06-04T14:32:05+00:00",
"@type":"http:\/\/www.w3.org\/2001\/XMLSchema#dateTime"
}
],
"http:\/\/schema.org\/dateModified":[
{
"@value":"2019-06-04T17:02:51+00:00",
"@type":"http:\/\/www.w3.org\/2001\/XMLSchema#dateTime"
}
],
"http:\/\/purl.org\/dc\/terms\/description":[
{
"@value":"mpermmbklmh",
"@language":"fa"
}
],
"http:\/\/purl.org\/dc\/terms\/created":[
{
"@value":"2015-10-15",
"@type":"http:\/\/www.w3.org\/2001\/XMLSchema#string"
},
{
"@value":"2015-10-15",
"@type":"http:\/\/www.w3.org\/2001\/XMLSchema#date"
}
],
"http:\/\/purl.org\/dc\/terms\/extent":[
{
"@value":"1 item",
"@type":"http:\/\/www.w3.org\/2001\/XMLSchema#string"
}
],
"http:\/\/pcdm.org\/models#memberOf":[
{
"@id":"http:\/\/future.islandora.ca\/node\/7"
}
],
"http:\/\/purl.org\/dc\/terms\/type":[
{
"@id":"http:\/\/future.islandora.ca\/taxonomy\/term\/3"
}
],
"http:\/\/purl.org\/dc\/terms\/subject":[
{
"@id":"http:\/\/future.islandora.ca\/taxonomy\/term\/27"
}
]
},
{
"@id":"http:\/\/future.islandora.ca\/en\/user\/1",
"@type":[
"http:\/\/schema.org\/Person"
]
},
{
"@id":"http:\/\/future.islandora.ca\/node\/7",
"@type":[
"http:\/\/pcdm.org\/models#Object"
]
},
{
"@id":"http:\/\/future.islandora.ca\/taxonomy\/term\/3",
"@type":[
"http:\/\/schema.org\/Thing"
]
},
{
"@id":"http:\/\/future.islandora.ca\/taxonomy\/term\/27",
"@type":[
"http:\/\/schema.org\/Thing"
]
}
]
}
```
2 changes: 2 additions & 0 deletions mkdocs.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ pages:
- 'Blocks': 'user-documentation/placing-blocks.md'
- 'Usage Stats': 'user-documentation/usage-stats.md'
- 'Multilingual': 'user-documentation/multilingual.md'
- Administrator Documentation:
- 'RDF Generation': 'islandora/rdf-mapping.md'
- Developer Documentation:
- REST Documentation:
- 'Introduction': 'technical-documentation/using-rest-endpoints.md'
Expand Down

0 comments on commit 2da9dbf

Please sign in to comment.