Development branch that merges loose ends together

.github/workflows/ci_tests.yml

@@ -29,22 +29,7 @@ jobs:
         --rcfile=pyproject.toml --disable=import-outside-toplevel,redefined-outer-name tests
 
       # safety-specific settings
-      run_safety: true
-      # 48547: RDFLib vulnerability: https://pyup.io/vulnerabilities/PVE-2022-48547/48547/
-      # 44715-44717: NumPy vulnerabilities:
-      #  https://pyup.io/vulnerabilities/CVE-2021-41495/44715/
-      #  https://pyup.io/vulnerabilities/CVE-2021-41496/44716/
-      #  https://pyup.io/vulnerabilities/CVE-2021-34141/44717/
-      # 70612: Jinja2 vulnerability. Only used as subdependency for mkdocs++ in tripper.
-      #  https://data.safetycli.com/v/70612/97c/
-      #  https://data.safetycli.com/v/72715/97c/   # update to mkdocs>=9.5.32
-      safety_options: |
-        --ignore=48547
-        --ignore=44715
-        --ignore=44716
-        --ignore=44717
-        --ignore=70612
-        --ignore=72715
+      run_safety: false
 
       ## Build package
       run_build_package: true
@@ -63,6 +48,16 @@ jobs:
       update_docs_landing_page: true
       package_dirs: tripper
 
+  safety:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@main
+      - name: Run Safety CLI to check for vulnerabilities
+        uses: pyupio/safety-action@v1
+        with:
+          api-key: ${{ secrets.SAFETY_API_KEY }}
+          args: --detailed-output # To always see detailed output from this action
+
   pytest:
     runs-on: ubuntu-latest
 

README.md

@@ -115,7 +115,7 @@ We gratefully acknowledge the following projects for supporting the development
 [Discovery of custom backends]: https://emmc-asbl.github.io/tripper/latest/backend_discovery/
 [tripper.datadoc]: https://emmc-asbl.github.io/tripper/latest/datadoc/introduction/
 [tripper.mappings]: https://emmc-asbl.github.io/tripper/latest/api_reference/mappings/mappings/
-[tripper.units]: https://emmc-asbl.github.io/tripper/latest/api_reference/units/units/
+[tripper.units]: https://emmc-asbl.github.io/tripper/latest/units/units/
 [tripper.convert]: https://emmc-asbl.github.io/tripper/latest/api_reference/convert/convert/
 [Known issues]: https://emmc-asbl.github.io/tripper/latest/known-issues/
 [MIT license]: https://emmc-asbl.github.io/tripper/latest/LICENSE/

docs/api_reference/datadoc/context.md

@@ -0,0 +1,3 @@
+# context
+
+::: tripper.datadoc.context

docs/datadoc/customisation.md

@@ -13,7 +13,7 @@ Tripper already include a default list of [predefined prefixes].
 Additional prefixed can be provided in two ways.
 
 ### With the `prefixes` argument
-Several functions in the API (like [save_dict()], [as_jsonld()] and [TableDoc.parse_csv()]) takes a `prefixes` argument with which additional namespace prefixes can provided.
+Several functions in the API (like [save_dict()], [told()] and [TableDoc.parse_csv()]) takes a `prefixes` argument with which additional namespace prefixes can provided.
 
 This may be handy when used from the Python API.
 
@@ -243,7 +243,7 @@ Instead, the list of available resource types should be stored and retrieved fro
 [predefined keywords]: keywords.md
 [default context]: https://raw.githubusercontent.com/EMMC-ASBL/tripper/refs/heads/master/tripper/context/0.2/context.json
 [save_dict()]: ../api_reference/datadoc/dataset.md#tripper.datadoc.dataset.save_dict
-[as_jsonld()]: ../api_reference/datadoc/dataset.md#tripper.datadoc.dataset.as_jsonld
+[told()]: ../api_reference/datadoc/dataset.md#tripper.datadoc.dataset.told
 [save_datadoc()]: ../api_reference/datadoc/dataset.md#tripper.datadoc.dataset.save_datadoc
 [TableDoc]: ../api_reference/datadoc/tabledoc.md/#tripper.datadoc.tabledoc.TableDoc
 [TableDoc.parse_csv()]: ../api_reference/datadoc/tabledoc.md/#tripper.datadoc.tabledoc.TableDoc.parse_csv

docs/datadoc/documenting-a-resource.md

@@ -46,13 +46,13 @@ We therefore have to define them explicitly
 !!! note "Side note"
 
     This dict is actually a [JSON-LD] document with an implicit context.
-    You can use [as_jsonld()] to create a valid JSON-LD document from it.
+    You can use [told()] to create a valid JSON-LD document from it.
     In addition to add a `@context` field, this function also adds some implicit `@type` declarations.
 
     ```python
     >>> import json
-    >>> from tripper.datadoc import as_jsonld
-    >>> d = as_jsonld(dataset, prefixes=prefixes)
+    >>> from tripper.datadoc import told
+    >>> d = told(dataset, prefixes=prefixes)
     >>> print(json.dumps(d, indent=4))  # doctest: +SKIP
     ```
 
@@ -89,7 +89,7 @@ Since the prefixes "sem" and "kb" are not included in the [Predefined prefixes],
 ```
 
 The returned `AttrDict` instance is an updated copy of `dataset` (casted to a dict subclass with attribute access).
-It correspond to a valid JSON-LD document and is the same as returned by [as_jsonld()].
+It correspond to a valid JSON-LD document and is the same as returned by [told()].
 
 You can use `ts.serialize()` to list the content of the triplestore (defaults to turtle):
 
@@ -216,7 +216,7 @@ The below example shows how to save all datasets listed in the CSV file [semdata
 [oteio:Generator]: https://w3id.org/emmo/domain/oteio/Generator
 [oteio:Parser]: https://w3id.org/emmo/domain/oteio/Parser
 [save_dict()]: ../api_reference/datadoc/dataset.md/#tripper.datadoc.dataset.save_dict
-[as_jsonld()]: ../api_reference/datadoc/dataset.md/#tripper.datadoc.dataset.as_jsonld
+[told()]: ../api_reference/datadoc/dataset.md/#tripper.datadoc.dataset.told
 [save_datadoc()]:
 ../api_reference/datadoc/dataset.md/#tripper.datadoc.dataset.save_datadoc
 [TableDoc]: ../api_reference/datadoc/tabledoc.md/#tripper.datadoc.tabledoc.TableDoc

docs/datadoc/keywords.md

@@ -1,6 +1,6 @@
 <!-- Do not edit! This file is generated with Tripper. Edit the keywords.yaml file instead. -->
 
-# Keywords for default
+# Keywords for domain: default
 The tables below lists the keywords the domain default.
 
 The meaning of the columns are as follows:
@@ -55,7 +55,7 @@ Resource published or curated by an agent.
 ## Properties on [Dataset]
 A collection of data, published or curated by an agent, and available for access or download in one or more representations.
 
-- subClassOf: [dcat:Resource]
+- subClassOf: [dcat:Resource], [emmo:EMMO_194e367c_9783_4bf5_96d0_9ad597d48d9a]
 
 | Keyword                | Range                                            | Conformance | Definition                                                                                                                 | Usage note                                                                                                    |
 | ---------------------- | ------------------------------------------------ | ----------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
@@ -125,6 +125,8 @@ A generator that can serialise an instance of a datamodel into a distribution.
 ## Properties on [Agent]
 Any entity carrying out actions with respect to the entities Catalogue and the Catalogued Resources.
 
+- subClassOf: [emmo:EMMO_2480b72b_db8d_460f_9a5f_c2912f979046]
+
 | Keyword      | Range                          | Conformance | Definition                                          | Usage note                                                                                                                                    |
 | ------------ | ------------------------------ | ----------- | --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
 | [name]       | [rdfs:Literal]<br>(xsd:string) | mandatory   | A name of the agent.                                |                                                                                                                                               |
@@ -169,6 +171,7 @@ A collection of operations that provides access to one or more datasets or data
 | [endpointURL]         | [rdfs:Literal]<br>(xsd:anyURI) | mandatory   | The root location or primary endpoint of the service (an IRI).                                          |            |
 | [endpointDescription] | [rdfs:Resource]                | recommended | A description of the services available via the end-points, including their operations, parameters etc. |            |
 | [servesDataset]       | [dcat:Dataset]                 | recommended | This property refers to a collection of data that this data service can distribute.                     |            |
+| [parser]              | [oteio:Parser]                 |             | A parser that can parse the distribution.                                                               |            |
 
 
 ## Properties on [DatasetSeries]
@@ -257,6 +260,10 @@ A standard or other specification to which a resource conforms.
 A media type, e.g. the format of a computer file.
 
 
+## Properties on [GenericResource]
+A generic resource.
+
+
 
 
 [Resource]: http://www.w3.org/ns/dcat#Resource
@@ -327,6 +334,7 @@ A media type, e.g. the format of a computer file.
 [statements]: https://w3id.org/emmo/domain/oteio#statement
 [rdfs:Literal]: http://www.w3.org/2000/01/rdf-schema#Literal
 [dcat:Resource]: http://www.w3.org/ns/dcat#Resource
+[emmo:EMMO_194e367c_9783_4bf5_96d0_9ad597d48d9a]: https://w3id.org/emmo#EMMO_194e367c_9783_4bf5_96d0_9ad597d48d9a
 [Dataset]: http://www.w3.org/ns/dcat#Dataset
 [distribution]: http://www.w3.org/ns/dcat#distribution
 [dcat:Distribution]: http://www.w3.org/ns/dcat#Distribution
@@ -405,6 +413,7 @@ A media type, e.g. the format of a computer file.
 [Generator]: https://w3id.org/emmo/domain/oteio#Generator
 [generatorType]: https://w3id.org/emmo/domain/oteio#generatorType
 [rdfs:Literal]: http://www.w3.org/2000/01/rdf-schema#Literal
+[emmo:EMMO_2480b72b_db8d_460f_9a5f_c2912f979046]: https://w3id.org/emmo#EMMO_2480b72b_db8d_460f_9a5f_c2912f979046
 [Agent]: http://xmlns.com/foaf/0.1/Agent
 [name]: http://xmlns.com/foaf/0.1/name
 [rdfs:Literal]: http://www.w3.org/2000/01/rdf-schema#Literal
@@ -431,6 +440,8 @@ A media type, e.g. the format of a computer file.
 [rdfs:Literal]: http://www.w3.org/2000/01/rdf-schema#Literal
 [servesDataset]: http://www.w3.org/ns/dcat#servesDataset
 [dcat:Dataset]: http://www.w3.org/ns/dcat#Dataset
+[parser]: https://w3id.org/emmo/domain/oteio#parser
+[oteio:Parser]: https://w3id.org/emmo/domain/oteio#Parser
 [dcat:Dataset]: http://www.w3.org/ns/dcat#Dataset
 [DatasetSeries]: http://www.w3.org/ns/dcat#DatasetSeries
 [Geometry]: http://www.w3.org/ns/locn#Geometry
@@ -506,3 +517,4 @@ A media type, e.g. the format of a computer file.
 [LegalResource]: http://data.europa.eu/eli/ontology#LegalResource
 [Standard]: http://purl.org/dc/terms/Standard
 [MediaType]: http://purl.org/dc/terms/MediaType
+[GenericResource]: http://www.w3.org/2000/01/rdf-schema#Resource

docs/datadoc/prefixes.md

@@ -11,13 +11,14 @@ See [User-defined prefixes] for how to extend this list with additional namespac
 | dctype  | http://purl.org/dc/dcmitype/                                      |
 | eli     | http://data.europa.eu/eli/ontology#                               |
 | foaf    | http://xmlns.com/foaf/0.1/                                        |
+| iana    | https://www.iana.org/assignments/media-types/                     |
 | locn    | http://www.w3.org/ns/locn#                                        |
 | odrl    | http://www.w3.org/ns/odrl/2/                                      |
 | owl     | http://www.w3.org/2002/07/owl#                                    |
 | prov    | http://www.w3.org/ns/prov#                                        |
 | rdf     | http://www.w3.org/1999/02/22-rdf-syntax-ns#                       |
 | rdfs    | http://www.w3.org/2000/01/rdf-schema#                             |
-| schema  | http://schema.org/                                                |
+| schema  | https://schema.org/                                               |
 | skos    | http://www.w3.org/2004/02/skos/core#                              |
 | spdx    | http://spdx.org/rdf/terms#                                        |
 | vcard   | http://www.w3.org/2006/vcard/ns#                                  |

docs/index.md

@@ -115,7 +115,7 @@ We gratefully acknowledge the following projects for supporting the development
 [Discovery of custom backends]: https://emmc-asbl.github.io/tripper/latest/backend_discovery/
 [tripper.datadoc]: https://emmc-asbl.github.io/tripper/latest/datadoc/introduction/
 [tripper.mappings]: https://emmc-asbl.github.io/tripper/latest/api_reference/mappings/mappings/
-[tripper.units]: https://emmc-asbl.github.io/tripper/latest/api_reference/units/units/
+[tripper.units]: https://emmc-asbl.github.io/tripper/latest/units/units/
 [tripper.convert]: https://emmc-asbl.github.io/tripper/latest/api_reference/convert/convert/
 [Known issues]: https://emmc-asbl.github.io/tripper/latest/known-issues/
 [MIT license]: https://emmc-asbl.github.io/tripper/latest/LICENSE/

mkdocs.yml

@@ -100,8 +100,8 @@ nav:
   - Units and quantities: units/units.md
   - Session: session.md
   - ... | api_reference/**
-  - Known issues: known-issues.md
   - For developers: developers.md
+  - Known issues: known-issues.md
   - Changelog: CHANGELOG.md
   - License: LICENSE.md
 

pyproject.toml

@@ -43,6 +43,7 @@ mappings = [
 ]
 datadoc = [
     "tripper[mappings]",
+    "PyLD>=2.0.0,<2.0.5",
     "PyYaml>=3.0.0,<6.0.3",  # todo: check lower version
     "requests>=2.0.0,<3.4.1",  # todo: check lower version
 ]
@@ -104,6 +105,8 @@ scripts_are_modules = true
 warn_unused_configs = true
 show_error_codes = true
 allow_redefinition = true
+#explicit-package-bases = true
+
 
 [tool.pylint]
 max-line-length = 80

tests/backends/test_sparqlwrapper_graphdb_fuseki.py

@@ -5,57 +5,13 @@
 https://emmc-asbl.github.io/tripper/latest/developers/.
 """
 
-import time
-
 import pytest
-import requests
 
 # URL to check if GraphDB is running.
 GRAPHDB_CHECK_URL = "http://localhost:7200/repositories"
 FUSEKI_CHECK_URL = "http://localhost:3030"
 
 
-def graphdb_available():
-    """
-    Help function that checks if the GraphDB instance is available.
-    If it is not, the tests that depend on it will be skipped.
-    """
-    timeout = 10  # seconds
-    interval = 1  # seconds
-    start_time = time.time()
-    while True:
-        try:
-            response = requests.get(GRAPHDB_CHECK_URL, timeout=timeout)
-            if response.status_code == 200:
-                return True
-        except requests.exceptions.RequestException:
-            pass
-
-        if time.time() - start_time > timeout:
-            return False
-        time.sleep(interval)
-
-
-def fuseki_available():
-    """
-    Help function that checks if the Fuseki instance is available
-    """
-    timeout = 10  # seconds
-    interval = 1  # seconds
-    start_time = time.time()
-    while True:
-        try:
-            response = requests.get(FUSEKI_CHECK_URL, timeout=timeout)
-            if response.status_code == 200:
-                return True
-        except requests.exceptions.RequestException:
-            pass
-
-        if time.time() - start_time > timeout:
-            return False
-        time.sleep(interval)
-
-
 def get_triplestore(tsname: str) -> "Triplestore":
     """Help function that returns a new triplestore object."""
     from tripper import Triplestore
@@ -240,7 +196,9 @@ def test_graphdb():
     Test the sparqlwrapper backend using GraphDB.
     """
     # Check if GraphDB is available and write a warning if it is not.
-    if not graphdb_available():
+    from tripper.utils import check_service_availability
+
+    if not check_service_availability(GRAPHDB_CHECK_URL, timeout=1):
         pytest.skip("GraphDB instance not available locally; skipping tests.")
 
     print("Testing graphdb")
@@ -252,7 +210,9 @@ def test_fuseki():
     Test the sparqlwrapper backend using Fuseki.
     """
     # Check if Fuseki is available and write a warning if it is not.
-    if not fuseki_available():
+    from tripper.utils import check_service_availability
+
+    if not check_service_availability(FUSEKI_CHECK_URL, timeout=1):
         pytest.skip("Fuseki instance not available locally; skipping tests.")
 
     print("Testing fuseki")