WIP

Backend to manage/serve a DCAT-AP catalogue whose resources are annotated with shapes (BYOS or automatically generated from data using shape-genaration-service); complient with the discovery specifification.

Microservices at play (see docker-compose.yml):

• dispatcher (config at dispatcher.ex) | entry point | microservice | |---|---|---| | json-api | resource | | shacl-gen | shape-generation | | paginated | dcat-serve-paginated |

• triplestore (or via database) Contains data of catalogues; dataset, distribution

• resource Exposes data at triplestore as api for DCAT... domain.lisp

• shape-generation Exposes api to generate shacl shapes based on a remote file/sparql endpoint.

• paginated Paginated api to explore catalogs (at http://localhost/paginated).

simulation of external sources

• demo-distribution-sparql Sample sparql endpoint (served at http://localhost:1042/sparql) to demo shape-generation and discovery (see docker-compose.dd.yml) Loads sample books.ttl.

• demo-distribution-files Serves sample files at http://localhost:1043 to demo shape-generation with (see docker-compose.dd.yml)

  • books.ttl ; used for source of sample distribution
  • water.trig ; sample external dcat source (used as well for discovery)

getting started + short demo

Clone the following github repos;

• shape-genaration-service
• dcat-serve-paginated-service
• dcat-discovery-service

as sisters to this repo. E.g. you'll have: (TODO; docker builds for microservices)

parent/ 
|---> ./shape-generation-service
|---> ./dcat-serve-paginated-service
|---> ./dcat-discovery-service
|---> ./app-dcat-catalog (this repo)

Before starting app, run the download.sh in shape-generation-service to the necessary binaries (in future should be delegated to build of docker image).

From withing this repo, run;

docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.dd.yml up -d
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.dd.yml down

Run the following python script to generate a new dataset with two distributions (uses json-api exposed by my-cl-resources); the externally hosted books.ttl and the external virtuoso endpoint.

./mwe.py

This will trigger shape generation. Afterwards, you can trigger reindexing of discovery service; the following reindexes the internal virtuoso endpoint (the first configured element).

curl -X POST http://localhost/discovery/reload \
  -H "Content-Type: application/vnd.api+json" \
  -d '{"id": "mu-internal"}'

To check out indexed shapes.

curl -H "Accept: text/plain" http://localhost/discovery/index

Or with json to get more detail.

To run a sample query, you can run the following;

curl -G http://localhost/discovery/search --data-urlencode "query@author.sparql"

To create some sample datasets/distributions for paginated, you can run

./paginated.py

Sample calls (using curl) for paginated API in paginated.sh.