Mandatory elements not checked for INSPIRE Discovery services (CSW)

[iuriemaxim]

This issue was discovered while trying to correctly index and harvest resources from a discovery service.
Even if the EC Validator did not indicated any issue, the National Romanian Geoportal indicated that some issues exist.

Investigating the problem, seems that some requirements from the TG for Discovery services are not checked by the EC INSPIRE validator and those are important ones that allow discovery services to be correctly accessed.

According to the TG for Discovery Services some elements are mandatory (M) and some need to be filled under certain circumstances (C). There are no optional elements (O).

However the validator is not triggering any error if these elements are present but not filled, or filled with invalid values (i.e. Fees - conditions applying to access and use other than those in INSPIRE Registry)

Test Conformance Class: Discovery Service - CSW is not triggering any error:.

Links to test:

Probably Correct - should not trigger errors and no errors are triggered:
https://inspire.meteoromania.ro/geonetwork/srv/eng/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetCapabilities

Sure Incorrect - should trigger errors, but currently no error is triggered:
https://inspire-staging.meteoromania.ro/geonetwork/srv/eng/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetCapabilities

For example the Title, Abstract and Fees (conditions applying to access and use) are not filled, while AccessConstraints (Limitation On Public Access) is ”none” instead of ”no limitations to public access” (label in the registry) or ”noLimitations” (camelCase value in the Registry) or ”no conditions apply” (TG MD version 1.3) or ”otherRestrictions”. Please also clarify which is the required text value, if any (secondary issue as most important is that the validator is not triggering errors for empty elements).

Probably the main issue is also present for view and download services, but we did not tested (no error triggered for empty elements).

But the secondary issue exist for view and download services as well as the value ”none” is not triggering any error if provided for Limitation On Public Access and conditions applying to access and use in the GetCapabilities document.

Links to test that have ”none” as value for fees and access constraints:
https://inspire-staging.meteoromania.ro/WIGOS/WFS?service=WFS&version=2.0.0&request=GetCapabilities
https://inspire-staging.meteoromania.ro/WIGOS/WMS?service=WMS&version=1.3.0&request=GetCapabilities

Iurie

[carlospzurita]

Dear @iuriemaxim

Thank you for noticing this and raising the issue.

We are checking the TG and the ATS as well for this. In principle, as the ATS states, the test only checks that the elements are present, but not the contents, following the Table 3 contents.

We are going to discuss internally if the ATS and subsequently the ETS need to be updated to solve this.

[iuriemaxim]

@carlospzurita Thank you for clarifications. I hope that if an element is mandatory, than it cant be provided empty (without any nillReason).

I would apreciate also if it can be provided an interpretation of the TG regarding the texts that are expected for the Fees and AccessConstraints.

[carlospzurita]

Dear @iuriemaxim

We have been discussing this issue, and the conclusion for this is to add checks for all the mandatory elements. However, given that the semantics of these elements can be rather complex to capture, we are just going to check that the elements is non-empty, and has the correct datatype.

This is going to apply only in the Scenario 2 described in the TG for Discovery Services, in which all elements are mapped to OGC ISO elements or described in the Extended Capabilities section. Scenario 1, the link to a Metadata URL, is going to remain unchanged.

[iuriemaxim]

@carlospzurita Thank you for the info.
Indded it is good at least to chek that the mandatory elements are filled.

Please consider if feaseable the following:

1. At least for the limitation on public access would be good to check the content against the values in the INSPIRE registry (https://inspire.ec.europa.eu/metadata-codelist/LimitationsOnPublicAccess)
   ... as I am expecting “none” value to fail the test. This would not be to difficult to be implemented as there are already such tests implemented in the validator (with hardcoded values).

If no tests will be implemented to check the content for Limitation on Public Access, then confirm that the values in the INSPIRE registry are not aplicable to this test for WCS and please clarify at least which is the required text value to indicate that no limitation on public access exist (for English language).

2. For the other mandatory/conditional elements, would be good to be consistent with the way the validator was implemented for other tests, by including manual checks, noticing the user to manually check the content of that specific field.

Can you please also provide more information regarding scenario 1, for a better understanding?

[AntoRot]

I agree that the validator should check that the mandatory elements are present and are not empty.
Concerning the OGC elements Fees (corresponding to Conditions applying to access and use) and Access constraints (corresponding to Limitations on public access), I think that the checks should be limited only to those above for the following reasons:

• the TG Requirement C.4 states that for the gmx:Anchor element the character string content of elements shall be provided in the language of the metadata. Currently the labels of the code lists values used for Limitations On Public Access and Conditions applying to access and use are only in English in the INSPIRE registry and consequently no common/controlled value exists for the languages other than English;
• probably linked to that, the validator only checks the value of the href attribute in the gmx:Anchor element and not the textual content;
• the OGC Web Services Common Standard that defines, inter alia, the requirements for the GetCapabilities response, states that for those elements "Reserved value NONE (case insensitive) shall be used to mean no fees or terms / no constraints are imposed".

It could be useful adding a cross-mapping of the values used in INSPIRE metadata and in the corresponding OGC elements in a future revision of the TG for the implementation of Discovery Services (to be done e.g. to align it to the latest version of the metadata TG).

[iuriemaxim]

@carlospzurita I see this modified for the 2021.0 milestone. Can you please provide some details?

[iuriemaxim]

@fabiovinci Two years ago this was marked as being under development. Now was set back to under analysis. Can you please provide more details to understand the changes in the status, as the need for development I think that is quite clear?

[fabiovinci]

Dear @iuriemaxim,

in the frame of the new contract "Operational support to the maintenance and technical evolution of INSPIRE components and artefacts", we are reviewing all the open issues assigned to people no more involved in the project, for this reason, we are re-analyzing this issue.

From a first check, it seems that at least the following improvement, in the case of scenario 2, can be applied:

to add checks for all the mandatory elements. However, given that the semantics of these elements can be rather complex to capture, we are just going to check that the elements are non-empty, and has the correct dataType.

Regarding the semantic check, before the implementation of any checks, a cross-mapping of the values used in INSPIRE metadata and in the corresponding OGC elements is needed. It could be included in a future revision of the TG for the implementation of Discovery Services (to be done e.g. to align it to the latest version of the metadata TG), as suggested by @AntoRot.

[fabiovinci]

Dear @dperezBM, @arantzaetxebarria,

could you please add a check, in the case of scenario 2, to verify that all mandatory metadata elements (Table 3 of the CSW TG) are filled in?

For example, for the following sample CSW (https://inspire-staging.meteoromania.ro/geonetwork/srv/eng/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetCapabilities) an error should be raised for the elements that are empty.