Update ndarray schema to 1.1 to fix issue 345

[braingram]

Fixes #345

This PR should be merged after #349 so the time schema errors fixed in that PR do not result in test failures once this PR fixes #345.

[WilliamJamieson]

Can you separate the change adding the new ndarray schema into a separate commit followed by all the supporting changes to make the history a bit easier to parse.

Also, can you post the diff between ndarray-1.0.0 and ndarray-1.1.0?

[braingram]

I may be missing something but I am unable to (locally) get the asdf-development tests to pass without updating the legacy converters to support the new schemas. Specifically:

• in astropy.io.misc.asdf)
  • table
  • column
• in asdf
  • integer
  • ndarray

The failing test is test_version_map_core_support[version6] which fails with a rather unhelpful error:
AttributeError: 'MockContext' object has no attribute '_warn_tag_mismatch'
The MockContext here, meant to mimic an AsdfFile does not have that method. If a method is added (that does nothing, matching the default behavior in AsdfFile), the error becomes a more helpful:

E           AssertionError: ASDF Standard version 1.6.0 requires support for tag:stsci.edu:asdf/core/integer-1.1.0, but no ExtensionType exists that explicitly supports that version.
E           assert 'tag:stsci.ed...integer-1.0.0' == 'tag:stsci.ed...integer-1.1.0'
E             - tag:stsci.edu:asdf/core/integer-1.1.0
E             ?                                   ^
E             + tag:stsci.edu:asdf/core/integer-1.0.0
E             ?                                   ^

If I then add a supported_versions class attribute to the IntegerType in asdf the test no longer fails for the integer tag.

However, the NDArrayType class does not appear to support a similar fix. If I add a supported_versions class attribute I get an even less helpful error at time of class definition:

TypeError: __class__ set to <class 'asdf.tags.core.ndarray.NDArrayType'> defining 'NDArrayType' as <class 'asdf.tags.core.ndarray.NDArrayType'>

To move this PR forward, am I correct that the asdf-development tests will fail until the converters are updated or am I missing something?

Also, thanks for the feedback on the history changes, I'll make them once the content of the PR is sorted out.

Aside from the examples in the description the diff for ndarray-1.0.0 and ndarray-1.1.0 is as follows:

>     oneOf:
>       - required: [source]
>       - required: [data]
>

[WilliamJamieson]

I may be missing something but I am unable to (locally) get the asdf-development tests to pass without updating the legacy converters to support the new schemas. Specifically:

• in astropy.io.misc.asdf)

  • table
  • column

• in asdf

  • integer
  • ndarray

The failing test is test_version_map_core_support[version6] which fails with a rather unhelpful error: AttributeError: 'MockContext' object has no attribute '_warn_tag_mismatch' The MockContext here, meant to mimic an AsdfFile does not have that method. If a method is added (that does nothing, matching the default behavior in AsdfFile), the error becomes a more helpful:

E           AssertionError: ASDF Standard version 1.6.0 requires support for tag:stsci.edu:asdf/core/integer-1.1.0, but no ExtensionType exists that explicitly supports that version.
E           assert 'tag:stsci.ed...integer-1.0.0' == 'tag:stsci.ed...integer-1.1.0'
E             - tag:stsci.edu:asdf/core/integer-1.1.0
E             ?                                   ^
E             + tag:stsci.edu:asdf/core/integer-1.0.0
E             ?                                   ^

If I then add a supported_versions class attribute to the IntegerType in asdf the test no longer fails for the integer tag.

However, the NDArrayType class does not appear to support a similar fix. If I add a supported_versions class attribute I get an even less helpful error at time of class definition:

TypeError: __class__ set to <class 'asdf.tags.core.ndarray.NDArrayType'> defining 'NDArrayType' as <class 'asdf.tags.core.ndarray.NDArrayType'>

To move this PR forward, am I correct that the asdf-development tests will fail until the converters are updated or am I missing something?

This should be paired with an asdf PR. Failures in astropy.io.misc.asdf are irrelevant.

[braingram]

Here is a partial diff between ndarray-1.0.0 and ndarray-1.1.0 (most differences are updating references/tags from 1.0 to 1.1):

374a386,389
>     oneOf:
>       - required: [source]
>       - required: [data]
>

[braingram]

There is 1 remaining reference to ndarray-1.0 in the asdf-schema-1.0.0 metaschema:

asdf-standard/resources/schemas/stsci.edu/asdf/asdf-schema-1.0.0.yaml

Line 39 in 1e22356

- $ref: "http://stsci.edu/schemas/asdf/core/ndarray-1.0.0#/definitions/datatype"

This references a portion of the ndarray-1.0.0 schema that is not affected by #345.

The CI failure of asdf-development tests is unavoidable as asdf does not yet have support for the new schemas (support will be added in this PR: asdf-format/asdf#1250).

I think it makes sense to merge this prior to the asdf PR so that the CI for that PR can hopefully have fewer errors (it will still fail some tests unless asdf-format/asdf#1247 is merged to allow xfailing certain new tags that are not supported in asdf).