which SHACL validators to try?

[VladimirAlexiev]

Requirements:

• must: SPARQLConstraint
• opt: fast and, or, not
• opt: sh:dataGraph so we can validate various stored models (consider options for using SHACL on graphs #140)
• opt: incremental validation

Let's define which validation engines to try.
We now have a pretty complete list at https://github.com/w3c-cg/awesome-semantic-shapes#readme and several contributors add to it often.

Here's a proposal:

• TopQuadrant API (Java & Jena) because it has been tested to date
  • ValiMate (part of the DNV suite) uses TopQuadrant API, so doesn't need to be tested separately
• Jena SHACL (Java) because it seems to have more recent development than TQ
• rdf4j ShaclSail (Java) because it can do incremental validation, and is used by GraphDB: see supported SHACL features
• pySHACL (Python) because a lot of TSOs/Power Engineers use Python, claims to have the most complete coverage of SHACL features, used by ModShape
• DataTreehouse/maplib (Rust) because it's fast for in-memory. However, it's not open source and DataTreehouse didn't agree to provide it for evaluation (Oct 2024)
• https://rudof-project.github.io/rudof/: a library and CLI that implements Shape Expressions, SHACL, DCTAP and other technologies in the RDF ecosystem (by the same guy who made SHACLEX, the first SHACL+SHEX implementation). It is implemented in Rust and it also provides Python bindings.

How about:

• JavaScript implementations?
• Shaclex, which implements SHACL and SHEX validation (don't know if that's relevant)

After we agree on the list, we need to research and list the limitations of every implementation.
This may eliminate some candidates.

@HarisVranaj please attach the presentation you showed 2d ago (I hope it's not confidential).
@griddigit-ci and @Sveino please comment on the proposal above, and I'll correct the list

[Sveino]

I agree on the requirement. First of all, we would like to have the UML/information model so that we really only need UML restriction. However, the world is more complicated. To avoid to have very technical UML/information model we will use a logical description of the constraints. This does not really need to be processes as is, but can be converted to relevant execution. This should be the primary motivation for not including SPARQL. Secondary is that we want to have engines that is optimised to execute well known constraints pattern.
So our primary test of the SHACL validation engines is to test our SHACL that we are applying rules that are not wrong understanding or bias to a particularly implementation.
I agree with the priorities and the argument for picking them. If we should add any addition, I would considered pySHACL. The reason for this is that a lot of TSOs are start using Python for Power Engineers. In addition Nick Car is a core developer. They have also boosted that they have the most complete coverage of the SHACL rules.

[HarisVranaj]

I have some suggestion from Erik for benchmarks in SHACL/SPARQL validator.

https://github.com/oxigraph/oxigraph/blob/main/bench/README.md
https://github.com/ad-freiburg/qlever/wiki/QLever-performance-evaluation-and-comparison-to-other-SPARQL-engines
Oxigraph is now optimised for memory usage (no longer using the rocksdb engine when using in memory) which on Erik's machine is 4 times faster then earlier versions (as this is including unzipping the file, real performance will even be better).

[VladimirAlexiev]

@HarisVranaj but do Oxigraph and QLever have SHACL implementations?
Please post links so I can include in https://github.com/VladimirAlexiev/awesome-semantic-shapes#shacl-validators and thereon to https://github.com/w3c-cg/awesome-semantic-shapes

[VladimirAlexiev]

Note to self: https://mail.google.com/mail/u/0/#sent/QgrcJHsTgsbXhdCJwNqzTbwQHVhdRXDHtBB asked Treehouse for access to maplib SHACL.

[VladimirAlexiev]

@Sveino points out that rdf4j 5.0.0 and 5.0.3 have some SHACL improvements:

• Support for sh:closed eclipse-rdf4j/rdf4j#1112
• SHACL - sh:equals, sh:lessThan, sh:lessThanEquals, sh:disjoint eclipse-rdf4j/rdf4j#4770
• SHACL - Support sh:name, sh:description, sh:defaultValue, sh:order, sh:group eclipse-rdf4j/rdf4j#4864
• SHACL - Improve logging eclipse-rdf4j/rdf4j#5027
• SHACL - Reduce duplicate use of Unique plan node eclipse-rdf4j/rdf4j#5028
• Allow configuration of threshold for using SPARQL validation approach when validation sh:maxCount eclipse-rdf4j/rdf4j#5108
• SHACL - sh:pattern validation results can differ between SPARQL vs. transactional validation eclipse-rdf4j/rdf4j#5112
• SHACL - Improve log message for internal exceptions eclipse-rdf4j/rdf4j#5113
• SHACL - Correctly handle missing values when validation SPARQL constraints eclipse-rdf4j/rdf4j#5114
• SHACL - bug where sh:class could cause missing validation results when nested sh:property and sh:node eclipse-rdf4j/rdf4j#5115
• GH-5130 sh:oneOrMorePath, sh:zeroOrMorePath, sh:zeroOrOnePath eclipse-rdf4j/rdf4j#5131
• SHACL - maxCount > 1 is slow for bulk validation eclipse-rdf4j/rdf4j#5153
• SHACL - rsx:targetShape throws exception when using multiple property shapes and complex paths eclipse-rdf4j/rdf4j#5167
• SHACL - validation of sh:qualifiedMaxCount may sometimes not when a statement matching the path is added eclipse-rdf4j/rdf4j#5139

Fixed in rdf4j 5.1.0

• SHACL - Validating maxCount/minCount across union of graphs with duplicates eclipse-rdf4j/rdf4j#5136
• SHACL - validation of sh:qualifiedMaxCount may sometimes not when a statement matching the path is added eclipse-rdf4j/rdf4j#5139
• SHACL - Improve performance of sh:pattern and make it more robust eclipse-rdf4j/rdf4j#5182
• SHACL - Add caching when "interning" Value objects eclipse-rdf4j/rdf4j#5189

Currently open SHACL issues (end-Jan 2025):

• SHACL W3C Compliance Tests eclipse-rdf4j/rdf4j#1117
• SHACL - Cleanup TempTest eclipse-rdf4j/rdf4j#1370
• SHACL - sh:xone eclipse-rdf4j/rdf4j#1488
• Validation of large datasets with SHACL eclipse-rdf4j/rdf4j#1914
• SHACL - Streaming validation results eclipse-rdf4j/rdf4j#2282
• SHACL - Batched revalidation eclipse-rdf4j/rdf4j#2283
• SHACL shape not recognized when not explicitly typed as a sh:NodeShape eclipse-rdf4j/rdf4j#2626
• Document use of ValidationException as generic way to get (remote or local) validation report eclipse-rdf4j/rdf4j#2641
• SHACL - Support pair wise constraints eclipse-rdf4j/rdf4j#2856
• SHACL - Extract AST into new module eclipse-rdf4j/rdf4j#2857
• SHACL - complete qualified value shapes implementation eclipse-rdf4j/rdf4j#2858
• SHACL - Benchmarks should penalize base sail access eclipse-rdf4j/rdf4j#2864
• SHACL - Optimize effective target chain when path can be ignored eclipse-rdf4j/rdf4j#2875
• SHACL - reimplement support for sh:detail eclipse-rdf4j/rdf4j#2885
• SHACL - Backwards-chaining RDFS connection should use statistics eclipse-rdf4j/rdf4j#3249
• ShaclSail should not split shapes by their targets eclipse-rdf4j/rdf4j#3584
• Shacl validation may leave threads running eclipse-rdf4j/rdf4j#3688
• SHACL - Split out bulk validation into separate method eclipse-rdf4j/rdf4j#4008
• SHACL: More descriptive message when a shape is both a sh:NodeShape and a sh:PropertyShape eclipse-rdf4j/rdf4j#4289
• SHACL - Possible NullPointerException when comparing two Select plan nodes eclipse-rdf4j/rdf4j#4313
• GH-4622 SHACL Validation Report preserve blank node IDs with remote repositories eclipse-rdf4j/rdf4j#4624
• SHACL - support binding query result variables in the sh:resultMessage when using SPARQL Constraint eclipse-rdf4j/rdf4j#4685
• SHACL - Check for expected variables in the SPARQL Constraint and SPARQL Target in a way so that users can recover their issue eclipse-rdf4j/rdf4j#4687
• SHACL - SPARQL Constraint and SPARQL Target should support variables with either of $ or ? eclipse-rdf4j/rdf4j#4688
• SHACL - Support the entire list of prioritized rules for which values should be used in the Validation report for SPARQL Constraint eclipse-rdf4j/rdf4j#4689
• GH-5107 shacl sail lost connection causes deadlock eclipse-rdf4j/rdf4j#5110
• SHACL - Support sh:oneOrMorePath, sh:zeroOrMorePath and sh:zeroOrOnePath eclipse-rdf4j/rdf4j#5130
• SHACL - Allow shapes to specify if they should be validated as transactional, legacy bulk or sparql bulk eclipse-rdf4j/rdf4j#5170

GraphDB will upgrade to rdf4j 5.0 at the end of the year.

[griddigit-ci]

When I tried pySHACL back in Jan and tried to package ModShape I has troubles. I was having performance issues. I was in touch with Nick at that time, there might be solutions, but I didn't have time to clean that up.

[HarisVranaj]

https://github.com/ad-freiburg/qlever
https://github.com/oxigraph/oxigraph/tree/main

[VladimirAlexiev]

@HarisVranaj Do Qlever and OxiGraph support SHACL? Please post links to documentation

[hmottestad]

I'm also working on supporting the last of the SHACL path expressions, and this should be included in RDF4J 5.1.0 or 5.2.0: eclipse-rdf4j/rdf4j#5131

I can also advertise that the RDF4J SHACL implementation supports incremental validation. If you have a large database and want to make a small change to your data, then the RDF4J SHACL engine will analyse your changes and only validate the affected target nodes.

[Sveino]

@hmottestad Very good. Incremental or difference validation is extremely relevant since we have a lot of SHACL rules that goes across multiple objects. The full graph is getting very big, and the changes are very limited. We have included the possiblity to exchange differences since 2005 using CIMXML/ RDFXML. We are not looking into how we can use JSON-LD to exchange this. See #53

[hmottestad]

RDF4J 5 has support for JSON-LD 1.1 with a customised version of Titanium JSON-LD that is considerably faster than the stock implementation that Jena is using.

I saw you were talking about DCAT, is your projected related to Datakatalogen på any chance?