Started getting a different message after updating to new xmlschema on Python 3.8: xmlschema.exceptions.XMLSchemaKeyError

[stanislaw]

Hello again,

This is a small follow-up to my previous ticket, see #360, where you helped me to make xmlschema work perfectly for my case of validating ReqIF XML files.

I haven't been updating my library for a few months, and now the GitHub CI is failing with the message that I am quoting below.

The further details:

• Locally with Python 3.7 (my lowest project baseline), I cannot reproduce it, even when updating to the latest 3.7 version.
• On GitHub CI it reproduces starting from 3.8. Failing job example: https://github.com/strictdoc-project/reqif/actions/runs/8796800137/job/24140347670?pr=162.

I am wondering if something substantial changed in how xmlschema works and prints validation messages between 3.7 and 3.8?

/home/runner/work/reqif/reqif/tests/integration/commands/validate/_xsd_schema_validation/03_invalid_table_internal/test.itest:2:8: error: CHECK: expected string not found in input
CHECK: Reason: Unexpected child with tag 'REQIF:IS-TABLE-INTERNAL' at position 1. Tag 'REQIF:OBJECT' expected.
       ^
<stdin>:1:1: note: scanning from here
Traceback (most recent call last):
^
<stdin>:56:46: note: possible intended match here
xmlschema.validators.exceptions.XMLSchemaParseError: missing group 'xhtml.BlkStruct.class':
                                             ^

Full input was:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/global_maps.py", line 301, in lookup_group
    obj = self.groups[qname]
KeyError: '{http://www.w3.org/1999/xhtml}xhtml.BlkStruct.class'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/groups.py", line 485, in _parse
    xsd_group = self.maps.lookup_group(self.name)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/global_maps.py", line 303, in lookup_group
    raise XMLSchemaKeyError(f'global xs:group {qname!r} not found')
xmlschema.exceptions.XMLSchemaKeyError: "global xs:group '{http://www.w3.org/1999/xhtml}xhtml.BlkStruct.class' not found"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/runner/work/reqif/reqif/reqif/cli/main.py", line 50, in <module>
    main()
  File "/home/runner/work/reqif/reqif/reqif/cli/main.py", line 42, in main
    ValidateCommand.execute(parser.get_validate_config())
  File "/home/runner/work/reqif/reqif/reqif/commands/validate/validate.py", line 54, in execute
    error_bundle = ValidateCommand._validate(config)
  File "/home/runner/work/reqif/reqif/reqif/commands/validate/validate.py", line 88, in _validate
    schema = xmlschema.XMLSchema(
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/schemas.py", line 523, in __init__
    self.maps.build()
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/global_maps.py", line 666, in build
    self.lookup_type(qname)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/global_maps.py", line 277, in lookup_type
    return cast(BaseXsdType, self._build_global(obj, qname, self.types))
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/global_maps.py", line 337, in _build_global
    global_map[qname] = factory_or_class(elem, schema)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/complex_types.py", line 91, in __init__
    super(XsdComplexType, self).__init__(elem, schema, parent, name)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 312, in __init__
    self.elem = elem
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 324, in __setattr__
    self._parse()
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/complex_types.py", line 153, in _parse
    self.content = self.schema.xsd_group_class(content_elem, self.schema, self)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/groups.py", line 117, in __init__
    super(XsdGroup, self).__init__(elem, schema, parent)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 312, in __init__
    self.elem = elem
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 324, in __setattr__
    self._parse()
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/groups.py", line 487, in _parse
    self.parse_error(_("missing group %r") % self.prefixed_name)
  File "/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/xmlschema/validators/xsdbase.py", line 197, in parse_error
    raise error
xmlschema.validators.exceptions.XMLSchemaParseError: missing group 'xhtml.BlkStruct.class':

Schema component:

  <xsd:group xmlns:xsd="http://www.w3.org/2001/XMLSchema" ref="xhtml.BlkStruct.class" />

Path: /xsd:schema/xsd:complexType[38]/xsd:group

Schema URL: file:///home/runner/work/reqif/reqif/reqif/reqif_schema/reqif.xsd

[brunato]

Hi,

The currently version of xmlschema does not support Python 3.7. The latest version that support Python 3.7 is v2.5.1.

About the failure something is changed with v3 for imports (it works with v2.5.1), I'm going to check if this is a bug or not.

thank you

[brunato]

This is a log of the failure with the current version:

>>> import xmlschema
>>> import logging
>>> logging.basicConfig()
>>> xs = xmlschema.XMLSchema11('reqif/reqif_schema/reqif.xsd', loglevel='debug')
DEBUG:xmlschema:Read schema from 'file:///home/brunato/tmp/reqif-main/reqif/reqif_schema/reqif.xsd'
DEBUG:xmlschema:Schema targetNamespace is 'http://www.omg.org/spec/ReqIF/20110401/reqif.xsd'
DEBUG:xmlschema:Declared namespaces: {'xml': 'http://www.w3.org/XML/1998/namespace', '': 'http://www.w3.org/1999/xhtml', 'REQIF': 'http://www.omg.org/spec/ReqIF/20110401/reqif.xsd', 'xsd': 'http://www.w3.org/2001/XMLSchema'}
DEBUG:xmlschema:Import namespace 'http://www.w3.org/1999/xhtml' from 'reqif/reqif_schema/www.omg.org/spec/ReqIF/20110402/driver.xsd'
DEBUG:xmlschema:<urlopen error [Errno 2] No such file or directory: '/home/brunato/tmp/reqif-main/reqif/reqif_schema/reqif/reqif_schema/www.omg.org/spec/ReqIF/20110402/driver.xsd'>

But it's the correct behavior, also lxml reports an error in this case:

>>> import lxml.etree as etree
>>> xs = etree.parse('reqif/reqif_schema/reqif.xsd')
>>> schema = etree.XMLS('reqif/reqif_schema/reqif.xsd')
etree.XMLSchema(                etree.XMLSchemaParseError(      etree.XMLSyntaxAssertionError(  
etree.XMLSchemaError(           etree.XMLSchemaValidateError(   etree.XMLSyntaxError(           
>>> schema = etree.XMLSchema(xs)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src/lxml/xmlschema.pxi", line 90, in lxml.etree.XMLSchema.__init__
lxml.etree.XMLSchemaParseError: Element '{http://www.w3.org/2001/XMLSchema}group', attribute 'ref': The QName value '{http://www.w3.org/1999/xhtml}xhtml.BlkStruct.class' does not resolve to a(n) model group definition., line 891

In xmlschema the changes have been done with commit f5d8450. This is in fact a fix because the previous behavior could cause unintended namespace load and break interoperability with other validators.

So, I'm sorry but you have to fix manually schema locations in your schema files or download them again (there is a new API download_schemas() and schema export has been improved in v3 releses).

[stanislaw]

Thanks you very much for a quick answer! I will get to testing this within this week and will let you know how it goes.

[stanislaw]

So, I'm sorry but you have to fix manually schema locations in your schema files or download them again (there is a new API download_schemas() and schema export has been improved in v3 releses).

I confirm that re-downloading schemas with Python 3.7+ solves the issue. Thank you!

strictdoc-project/reqif#165