Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LU Validation Test for ExistingLandUse reports wrong dateTime Format #424

Closed
robblis opened this issue Oct 16, 2020 · 9 comments
Closed
Labels
deployed in reference validator Solution deployed in production
Milestone

Comments

@robblis
Copy link

robblis commented Oct 16, 2020

Hi Validator Community,

I'm running validation tests for the LU theme using the ExistingLandUse feature type.

I am getting the following error reported in the Conformance Class 'Data consistency' test.

For all features verify that either
validFrom or validTo are missing or nil or
validTo is not before the value of validFrom.

Relevant requirements:
IR Requirement Article 12 (3): Other Requirements and Rules. Where the attributes validFrom and validTo are used, the value of validTo shall not be before the value of validFrom.

Source: Abstract Test Case 'Temporal consistency', INSPIRE Data Specification Template, A.3.4

Status Failed
Duration 0.004 s
Assertion ID EIDcefa9614-ce69-43d4-bed5-7758e7d890a9
Assertion URI Link

Messages:
System error in the Executable Test Suite. Please contact a system administrator. Error information:
[err:FORG0001] Wrong xs:dateTime format: '1943-08-14' (try e.g. '2000-12-31T23:59:59.999').
(621/153)

However, the data type for the 'validFrom' and the 'validTo' in the ExistingLandUse.xsd is specified as a 'date' type and not a 'dateTime' type as shown in the xml snippet below from taken from the ExistingLandUse.xsd.

-- Definition -- The time when the phenomenon started to exist in the real world. -- Definition -- The time from which the phenomenon no longer exists in the real world.

I've attached the test xml.
ExistingLandUse_service_xml.zip

@iuriemaxim
Copy link

iuriemaxim commented Oct 16, 2020

Hello @robblis The Data Specification for LU specifies at page 22 (34 in pdf)

image

So according to this section of the TG they should be DateTime. Aren't these the elements referred in your issue?

Indeed at page 50 (62 in PDF) the type is Date while the definition is mentioning Time.

image

Only for this feature type one definition seems to correspond to the data type

image

Probably the TWG did not properly updated the Data Specifications TG when they decided to switch from DateTime to Date.

They updated the application schema, but not updated the TG accordingly.

image

So indeed the validator should not trigger an error and a new Corrigendum should be added here:
https://inspire.ec.europa.eu/id/document/tg/lu

@carlospzurita
Copy link
Contributor

We have checked the data specification that @iuriemaxim has linked, and it is indeed confusing the usage of Date in validFrom and validTo, in contrast with the attributes beginLifespanVersion and endLifespanVersion.

Also, checking the INSPIRE generic model we get in the Recommendation 17 that this attributes should be specified using DateTime.

imagen

This is of course a recommendation and it should not be enforced, but we are going to check internally in which way should this issue be treated, if the test should be relaxed admitting Date and DateTime or move to a DateTime requirement

@iuriemaxim
Copy link

@carlospurtitsa When analysing ”if the test should be relaxed admitting Date and DateTime or move to a DateTime requirement”, please take into consideration that the Land Use phenomenons are difficult to be indicated even by date, as by hour minute and second will be even strange.

Imagine a sentence like: ”this area is used for constructions starting from 2000-12-31T23:59:59.999, before being used as a pasture”.

Most probably this should be clarified with the TWG that wrote the DS TG for LU, as most probably they changed the dateTime to Date.

The same problem we had in the TWG dealing with SD, HB and BR Data themes.
Just that we did not used validFrom and validTo for the species collecting Phenomena (SD) in order to overpass the Recommendation 17.

image

Same for habitats and biotopes (HB):

image

@robblis
Copy link
Author

robblis commented Oct 19, 2020

Hi @carlospzurita

I am also testing the LU theme.

I found the same validation issue with the plu:validFrom and plu:validTo for plu:SpatialPlan and plu:SupplementaryRegulation feature in the plu/PlannedLandUse.xsd.

XPath:
plu:SpatialPlan/plu:validFrom
plu:SpatialPlan/plu:validTo

plu:SupplementaryRegulation/plu:validFrom
plu:SupplementaryRegulation/plu:validTo

Thanks,
Robert

LU_plu_SupplementaryRegulation.zip

LU_plu_SpatialPlan.zip

@iuriemaxim
Copy link

iuriemaxim commented Oct 19, 2020

@robblis @carlospzurita
The validFrom and ValidTo elements are defined as Date type in the following feature types, both in the XSD schemas and in the DS TG for LU:

  • ExistingLandUseDataSet
  • ExistingLandUseObject
  • SampledExistingLandUseDataSet
  • ExistingLandUseSample
  • ExistingLandUseGrid
  • SpatialPlan
  • ZoningElement
  • SupplementaryRegulation

@carlospzurita
Copy link
Contributor

Dear @robblis @iuriemaxim

We modified the xQuery functiones that were used to validate the validFrom and validTo elements. Even if those elements were valid against the schema, there were implemented additional checks for them to ensure that validFrom was set before validTo. To do this, they were both treated as DateTime. Now both Dates and DateTimes are allowed to define this elements.

Please have a look into the staging instance

@carlospzurita carlospzurita added ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing and removed under analysis labels Oct 22, 2020
@iuriemaxim
Copy link

@carlospzurita I think that only Date should be correct. Otherwise if dateTime will be used, the GML will be validated by the validator, but any XML/GML validator will trigger an error due to the fact that the schema is indicating that the element should be Date.
As @MarcoMinghini mentioned in the #402, schemas are not usually changed unless there are clear reasons to correct some mistakes.

Unless the schema will be changed please reconsider the decision in such a way to trigger an error if the elements are provided as DateTime as any GML/XML validator will do the same.

@carlospzurita
Copy link
Contributor

Dear @iuriemaxim

The modification made on the ETS does not overrides the schema validation. The change was made to be able to check that the time consistency between validFrom and validTo is maintained, either if they are Date or Datetime. If the schema does not support the element as Datetime, the schema validation will fail.

@carlospzurita carlospzurita added this to the v2021.0 milestone Nov 3, 2020
@robblis
Copy link
Author

robblis commented Nov 5, 2020

Ran new test on modified validator. Tests passed.

@robblis robblis closed this as completed Nov 5, 2020
@carlospzurita carlospzurita added solved Solution developed and accepted, not yet deployed and removed ready for testing Solution provided to reporter or developed & deployed in staging (or beta), waiting for testing labels Nov 13, 2020
@carlospzurita carlospzurita reopened this Nov 13, 2020
@carlospzurita carlospzurita added deployed in reference validator Solution deployed in production and removed solved Solution developed and accepted, not yet deployed labels Jan 26, 2021
@carlospzurita carlospzurita modified the milestones: v2021.0, v2021.1 Feb 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed in reference validator Solution deployed in production
Projects
None yet
Development

No branches or pull requests

3 participants