Local and endpoint migration for renamed and removed entities. #554
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| prefix gist: <https://ontologies.semanticarts.com/gist/> | ||
|
|
||
| DELETE { | ||
| graph ?graph { | ||
| ?ref ?refProp ?oldClass . | ||
| ?oldClass ?defProp ?defObj . | ||
| } | ||
| } | ||
| INSERT { | ||
| graph ?graph { | ||
| ?ref ?refProp ?newClass . | ||
| ?newClass ?defProp ?defObj . | ||
| } | ||
| } | ||
| where { | ||
| values (?oldClass ?newClass) { | ||
| # Issue #434 | ||
| (gist:MimeType gist:MediaType) | ||
| # Issue #483 | ||
| (gist:BuildingAddress gist:StreetAddress) | ||
| } | ||
| graph ?graph { | ||
| { | ||
| ?ref ?refProp ?oldClass | ||
| } UNION { | ||
| ?oldClass ?defProp ?defObj | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| prefix gist: <https://ontologies.semanticarts.com/gist/> | ||
|
|
||
| DELETE { | ||
| ?ref ?refProp ?oldClass . | ||
| ?oldClass ?defProp ?defObj . | ||
| } | ||
| INSERT { | ||
| ?ref ?refProp ?newClass . | ||
| ?newClass ?defProp ?defObj . | ||
| } | ||
| where { | ||
| values (?oldClass ?newClass) { | ||
| # Issue #434 | ||
| (gist:MimeType gist:MediaType) | ||
| # Issue #483 | ||
| (gist:BuildingAddress gist:StreetAddress) | ||
| } | ||
| { | ||
| ?ref ?refProp ?oldClass | ||
| } UNION { | ||
| ?oldClass ?defProp ?defObj | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| prefix gist: <https://ontologies.semanticarts.com/gist/> | ||
|
|
||
| DELETE { | ||
| graph ?graph { | ||
| ?s ?oldProp ?o . | ||
| ?ref ?refProp ?oldProp . | ||
| ?oldProp ?defProp ?defObj . | ||
| } | ||
| } | ||
| INSERT { | ||
| graph ?graph { | ||
| ?s ?newProp ?o . | ||
| ?ref ?refProp ?newProp . | ||
| ?newProp ?defProp ?defObj . | ||
| } | ||
| } | ||
| where { | ||
| values (?oldProp ?newProp) { | ||
| # Issue #171 | ||
| (gist:decimalValue gist:numericValue) | ||
| # Issue #126 | ||
| (gist:networkConnection gist:links) | ||
| (gist:hasFromNode gist:linksFrom) | ||
| (gist:hasToNode gist:linksTo) | ||
| # Issue #483 | ||
| (gist:hasStreetAddress gist:hasAddress) | ||
| # Issue 188 | ||
| (gist:directlyPrecedes gist:precedesDirectly) | ||
| (gist:hasUoM gist:hasUnitOfMeasure) | ||
| (gist:multiplicand gist:hasMultiplicand) | ||
| (gist:multiplier gist:hasMultiplier) | ||
| (gist:affectedBy gist:isAffectedBy) | ||
| (gist:allocatedBy gist:isAllocatedBy) | ||
| (gist:denominator gist:hasDenominator) | ||
| (gist:numerator gist:hasNumerator) | ||
| (gist:categorizedBy gist:isCategorizedBy) | ||
| (gist:actualStart gist:hasActualStart) | ||
| (gist:actualEnd gist:hasActualEnd) | ||
| (gist:expressedIn gist:isExpressedIn) | ||
| (gist:plannedStart gist:hasPlannedStart) | ||
| (gist:plannedEnd gist:hasPlannedEnd) | ||
| (gist:triggeredBy gist:isTriggeredBy) | ||
| (gist:hasJurisdiction gist:hasJurisdictionOver) | ||
| (gist:governedBy gist:isGovernedBy) | ||
| (gist:recognizedBy gist:isRecognizedBy) | ||
| (gist:characterizedAs gist:isCharacterizedAs) | ||
| (gist:start gist:hasStart) | ||
| (gist:end gist:hasEnd) | ||
| (gist:fromPlace gist:comesFromPlace) | ||
| (gist:toPlace gist:goesToPlace) | ||
| (gist:geoContains gist:containsGeographically) | ||
| (gist:timeZoneStandardUsed gist:usesTimeZoneStandard) | ||
| (gist:identifiedBy gist:isIdentifiedBy) | ||
| (gist:permanentGeoOccupies gist:occupiesGeographicallyPermanently) | ||
| (gist:offspringOf gist:hasBiologicalParent) | ||
| (gist:fromAgent gist:comesFromAgent) | ||
| (gist:toAgent gist:goesToAgent) | ||
| (gist:memberOf gist:isMemberOf) | ||
| (gist:hasGetter gist:hasRecipient) | ||
| (gist:directlyPrecededBy gist:followsDirectly) | ||
| (gist:offspringOf gist:hasBiologicalParent) | ||
| (gist:occursAt gist:occursIn) | ||
| (gist:madeUpOf gist:isMadeUpOf) | ||
| (gist:convertToBase gist:baseConversionFactor) | ||
| (gist:renderedOn gist:isRenderedOn) | ||
| (gist:basisFor gist:isBasisFor) | ||
| (gist:hasTag gist:tagText) | ||
| (gist:connectedTo gist:isConnectedTo) | ||
| (gist:about gist:isAbout) | ||
| (gist:actual gist:hasActual) | ||
| (gist:aspectOf gist:isAspectOf) | ||
| (gist:basedOn gist:isBasedOn) | ||
| (gist:convertToStandard gist:standardConversionFactor) | ||
| (gist:describedIn gist:isDescribedIn) | ||
| (gist:directPartOf gist:isDirectPartOf) | ||
| (gist:directSubTaskOf gist:isDirectSubtaskOf) | ||
| (gist:directlyRecognizedBy gist:isRecognizedDirectlyBy) | ||
| (gist:geoContainedIn gist:isGeographicallyContainedIn) | ||
| (gist:geoOccupiedBy gist:isGeographicallyOccupiedBy) | ||
| (gist:parentOf gist:hasBiologicalOffspring) | ||
| (gist:partOf gist:isPartOf) | ||
| (gist:planned gist:hasPlanned) | ||
| (gist:subTaskOf gist:isSubTaskOf) | ||
| (gist:viableRange gist:hasViableRange) | ||
| (gist:permanentGeoOccupiedBy gist:isGeographicallyPermanentlyOccupiedBy) | ||
| (gist:recordedOn gist:isRecordedAt) | ||
| (gist:sameTimeAs gist:isSameTimeAs) | ||
| (gist:geoOccupies gist:occupiesGeographically) | ||
| (gist:unitSymbolHTML gist:unitSymbolHtml) | ||
| (gist:lastModifiedOn gist:wasLastModifiedAt) | ||
| (gist:offsetToUniversal gist:hasOffsetToUniversal) | ||
| } | ||
| graph ?graph { | ||
| { | ||
| ?s ?oldProp ?o | ||
| } UNION { | ||
| ?ref ?refProp ?oldProp | ||
| } UNION { | ||
| ?oldProp ?defProp ?defObj | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| prefix gist: <https://ontologies.semanticarts.com/gist/> | ||
|
|
||
| DELETE { | ||
| ?s ?oldProp ?o . | ||
| ?ref ?refProp ?oldProp . | ||
| ?oldProp ?defProp ?defObj . | ||
| } | ||
| INSERT { | ||
| ?s ?newProp ?o . | ||
| ?ref ?refProp ?newProp . | ||
| ?newProp ?defProp ?defObj . | ||
| } | ||
| where { | ||
| values (?oldProp ?newProp) { | ||
| # Issue #171 | ||
| (gist:decimalValue gist:numericValue) | ||
| # Issue #126 | ||
| (gist:networkConnection gist:links) | ||
| (gist:hasFromNode gist:linksFrom) | ||
| (gist:hasToNode gist:linksTo) | ||
| # Issue #483 | ||
| (gist:hasStreetAddress gist:hasAddress) | ||
| # Issue 188 | ||
| (gist:directlyPrecedes gist:precedesDirectly) | ||
| (gist:hasUoM gist:hasUnitOfMeasure) | ||
| (gist:multiplicand gist:hasMultiplicand) | ||
| (gist:multiplier gist:hasMultiplier) | ||
| (gist:affectedBy gist:isAffectedBy) | ||
| (gist:allocatedBy gist:isAllocatedBy) | ||
| (gist:denominator gist:hasDenominator) | ||
| (gist:numerator gist:hasNumerator) | ||
| (gist:categorizedBy gist:isCategorizedBy) | ||
| (gist:actualStart gist:hasActualStart) | ||
| (gist:actualEnd gist:hasActualEnd) | ||
| (gist:expressedIn gist:isExpressedIn) | ||
| (gist:plannedStart gist:hasPlannedStart) | ||
| (gist:plannedEnd gist:hasPlannedEnd) | ||
| (gist:triggeredBy gist:isTriggeredBy) | ||
| (gist:hasJurisdiction gist:hasJurisdictionOver) | ||
| (gist:governedBy gist:isGovernedBy) | ||
| (gist:recognizedBy gist:isRecognizedBy) | ||
| (gist:characterizedAs gist:isCharacterizedAs) | ||
| (gist:start gist:hasStart) | ||
| (gist:end gist:hasEnd) | ||
| (gist:fromPlace gist:comesFromPlace) | ||
| (gist:toPlace gist:goesToPlace) | ||
| (gist:geoContains gist:containsGeographically) | ||
| (gist:timeZoneStandardUsed gist:usesTimeZoneStandard) | ||
| (gist:identifiedBy gist:isIdentifiedBy) | ||
| (gist:permanentGeoOccupies gist:occupiesGeographicallyPermanently) | ||
| (gist:offspringOf gist:hasBiologicalParent) | ||
| (gist:fromAgent gist:comesFromAgent) | ||
| (gist:toAgent gist:goesToAgent) | ||
| (gist:memberOf gist:isMemberOf) | ||
| (gist:hasGetter gist:hasRecipient) | ||
| (gist:directlyPrecededBy gist:followsDirectly) | ||
| (gist:offspringOf gist:hasBiologicalParent) | ||
| (gist:occursAt gist:occursIn) | ||
| (gist:madeUpOf gist:isMadeUpOf) | ||
| (gist:convertToBase gist:baseConversionFactor) | ||
| (gist:renderedOn gist:isRenderedOn) | ||
| (gist:basisFor gist:isBasisFor) | ||
| (gist:hasTag gist:tagText) | ||
| (gist:connectedTo gist:isConnectedTo) | ||
| (gist:about gist:isAbout) | ||
| (gist:actual gist:hasActual) | ||
| (gist:aspectOf gist:isAspectOf) | ||
| (gist:basedOn gist:isBasedOn) | ||
| (gist:convertToStandard gist:standardConversionFactor) | ||
| (gist:describedIn gist:isDescribedIn) | ||
| (gist:directPartOf gist:isDirectPartOf) | ||
| (gist:directSubTaskOf gist:isDirectSubtaskOf) | ||
| (gist:directlyRecognizedBy gist:isRecognizedDirectlyBy) | ||
| (gist:geoContainedIn gist:isGeographicallyContainedIn) | ||
| (gist:geoOccupiedBy gist:isGeographicallyOccupiedBy) | ||
| (gist:parentOf gist:hasBiologicalOffspring) | ||
| (gist:partOf gist:isPartOf) | ||
| (gist:planned gist:hasPlanned) | ||
| (gist:subTaskOf gist:isSubTaskOf) | ||
| (gist:viableRange gist:hasViableRange) | ||
| (gist:permanentGeoOccupiedBy gist:isGeographicallyPermanentlyOccupiedBy) | ||
| (gist:recordedOn gist:isRecordedAt) | ||
| (gist:sameTimeAs gist:isSameTimeAs) | ||
| (gist:geoOccupies gist:occupiesGeographically) | ||
| (gist:unitSymbolHTML gist:unitSymbolHtml) | ||
| (gist:lastModifiedOn gist:wasLastModifiedAt) | ||
| (gist:offsetToUniversal gist:hasOffsetToUniversal) | ||
| } | ||
| { | ||
| ?s ?oldProp ?o | ||
| } UNION { | ||
| ?ref ?refProp ?oldProp | ||
| } UNION { | ||
| ?oldProp ?defProp ?defObj | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,67 @@ | ||||||
| prefix skos: <http://www.w3.org/2004/02/skos/core#> | ||||||
| prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> | ||||||
| prefix sh: <http://www.w3.org/ns/shacl#> | ||||||
| PREFIX owl: <http://www.w3.org/2002/07/owl#> | ||||||
| prefix gist: <https://ontologies.semanticarts.com/gist/> | ||||||
|
|
||||||
| CONSTRUCT { | ||||||
| ?report a sh:ValidationReport ; | ||||||
| sh:conforms false ; | ||||||
| sh:result | ||||||
| [ | ||||||
| a sh:ValidationResult ; | ||||||
| sh:focusNode ?deleted ; | ||||||
| sh:resultMessage ?error ; | ||||||
| sh:resultSeverity sh:Warning ; | ||||||
| sh:sourceConstraintComponent <urn:constraint:removed-entity> | ||||||
| ] . | ||||||
| } | ||||||
| WHERE { | ||||||
| { | ||||||
| select distinct ?deleted where { | ||||||
| values ?deleted { | ||||||
| gist:PhysicalThing | ||||||
| gist:Room | ||||||
| gist:SocialBeing | ||||||
| gist:TemmplateTask | ||||||
sa-bpelakh marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| gist:TimeInterval | ||||||
| gist:Weight | ||||||
| gist:_unitedNations | ||||||
| gist:date | ||||||
| gist:dateTime | ||||||
| gist:geoDirectlyContainedIn | ||||||
| gist:geoDirectlyContains | ||||||
| gist:getter | ||||||
| gist:giver | ||||||
| gist:hasA | ||||||
| gist:localDate | ||||||
| gist:hasA | ||||||
sa-bpelakh marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| gist:localDate | ||||||
| gist:localDateTime | ||||||
| gist:of | ||||||
| gist:party | ||||||
| gist:sameOrderAs | ||||||
| gist:strictlyPrecededBy | ||||||
| gist:strictlyPrecedes | ||||||
| gist:time | ||||||
| gist:universalDate | ||||||
| gist:universalDataTime | ||||||
| gist:universalTime | ||||||
| gist:streetAddressOf | ||||||
| gist:communicationAddressOf | ||||||
| gist:OrdinalCollection | ||||||
| gist:OrdinalMember | ||||||
| } | ||||||
| { | ||||||
| ?s ?deleted ?o | ||||||
| } UNION { | ||||||
| ?s ?p ?deleted | ||||||
| } UNION { | ||||||
| ?deleted ?p ?o | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
| bind(<urn:deleted-entity-vaidation-report> as ?report) | ||||||
|
There was a problem hiding this comment. Minor typo:
Suggested change
|
||||||
| bind(CONCAT("Removed entity gist:", | ||||||
| REPLACE(STR(?deleted), '^.*[/#]', ''), " referenced in data.") as ?error) | ||||||
| } | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| bundle: gist-v10.0-migration | ||
| variables: | ||
| migration: "./migration" | ||
| endpoint: "<SPARQL_UPDATE_URI>" | ||
| update_endpoint: "{endpoint}" | ||
| user: "UPDATE_USER" | ||
| password: "UPDATE_PASSWORD" | ||
| report: "./migration-report" | ||
| actions: | ||
| - action: "sparql" | ||
| message: "Migrate renamed classes in named graphs." | ||
| query: "{migration}/v10.0/action_rename_classes.rq" | ||
| endpoint: | ||
| query_uri: "{endpoint}" | ||
| update_uri: "{update_endpoint}" | ||
| user: "{user}" | ||
| password: "{password}" | ||
| - action: "sparql" | ||
| message: "Migrate renamed properties in named graphs." | ||
| query: "{migration}/v10.0/action_rename_properties.rq" | ||
| endpoint: | ||
| query_uri: "{endpoint}" | ||
| update_uri: "{update_endpoint}" | ||
| user: "{user}" | ||
| password: "{password}" | ||
| - action: "sparql" | ||
| message: "Migrate renamed classes in default graph." | ||
| query: "{migration}/v10.0/action_rename_classes_default.rq" | ||
| endpoint: | ||
| query_uri: "{endpoint}" | ||
| update_uri: "{update_endpoint}" | ||
| user: "{user}" | ||
| password: "{password}" | ||
| - action: "sparql" | ||
| message: "Migrate renamed properties in default graph." | ||
| query: "{migration}/v10.0/action_rename_properties_default.rq" | ||
| endpoint: | ||
| query_uri: "{endpoint}" | ||
| update_uri: "{update_endpoint}" | ||
| user: "{user}" | ||
| password: "{password}" | ||
| - action: "verify" | ||
| message: "Find remaining problems" | ||
| type: "construct" | ||
| inference: "none" | ||
| stopOnFail: false | ||
| target: '{report}' | ||
| endpoint: | ||
| query_uri: "{endpoint}" | ||
| user: "{user}" | ||
| password: "{password}" | ||
| queries: | ||
| source: '{migration}/v10.0' | ||
| includes: | ||
| - 'detect_*.rq' |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we have both
action_rename_classes.rqandaction_rename_classes_default.rqand similarly for properties?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do renaming in two steps:
We do it in this manner because the default graph does not have a universally recognized reference URI, and in some triple stores it doesn't have a URI at all. Also, when processing local file data, there are no named graphs, so only the default queries (which don't specify a graph) are used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A local .trig or N-quads file can have named graphs. I assume that works correctly for them as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Jamie-SA There is no current support in onto-tool for .trig or .nq inputs. Do you view this as a deal breaker for this functionality? There is no way I can code and test this before Monday
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is not a deal breaker for me, I had assumed the library you use to read/write would support all of the main formats. But if not, I wouldn't hold this up because of it.
I have switched almost entirely to .trig because of it's support for named graphs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The toolkit (
rdflib) does support it, but I have not enabled it - and there is an existing issue (semanticarts/ontology-toolkit#65), to which I will give some attention after this release.