validation handling of `visit_repeat_nodes` results in incorrect validation

[braingram]

I will work on a minimum example that can be turned into a unit test.

The issue came up when astropy/asdf-astropy#270 introduced the table-1.3.0 schema which uses tag validators:
https://github.com/astropy/asdf-astropy/blob/8d798e24c2521e9646cb2d41e8c6487f96dbb611/asdf_astropy/resources/schemas/table/table-1.3.0.yaml#L103
for checking columns instead of $refs to column-like schemas as is done in table-1.2.0:
https://github.com/astropy/asdf-astropy/blob/8d798e24c2521e9646cb2d41e8c6487f96dbb611/asdf_astropy/resources/schemas/table/table-1.2.0.yaml#L103

When a table contains a column with an invalid unit (in this case a magunit which isn't supported which is a separate bug) the validator bug pops up. When a magunit is serialized it produced a non-unit-schema conforming representation (maybe also a separate bug). This results in a tagged tree for the table that should be invalid since it contains a {'unit': '3.6307805e-20erg.Hz**-1.s**-1.cm**-2'} where a string should be. However during validation asdf incorrectly marks this as valid because:

• the column instance and column schema are validated as part of the table anyOf where the validation fails but asdf marks the instance/schema pair as "seen"
• then when the column instance and column schema are validated due to the instance having the column tag validated is skipped because the instance/schema pair was "seen" above

We can't just revisit all nodes (since that produces recursion errors) but if we remove that check and revisit nodes (for testing):
https://github.com/asdf-format/asdf/actions/runs/21912149854/job/63268918611?pr=1999#step:10:2491
we see the error:

[79](https://github.com/asdf-format/asdf/actions/runs/21912149854/job/63268918611?pr=1999#step:10:2480)
E           Failed validating 'anyOf' in schema['properties']['columns']['items']:
E               {'anyOf': [{'$ref': 'http://stsci.edu/schemas/asdf/core/column-1.0.0'},
E                          {'$ref': 'http://stsci.edu/schemas/asdf/table/column-1.1.0'},
E                          {'$ref': 'http://stsci.edu/schemas/asdf/core/ndarray-1.1.0'},
E                          {'$ref': 'http://stsci.edu/schemas/asdf/time/time-1.2.0'},
E                          {'$ref': 'http://stsci.edu/schemas/asdf/unit/quantity-1.2.0'},
E                          {'$ref': '../coordinates/skycoord-1.0.0'},
E                          {'$ref': '../coordinates/earthlocation-1.1.0'},
E                          {'$ref': '../time/timedelta-1.1.0'},
E                          {'$ref': 'ndarraymixin-1.0.0'}]}
E           
E           On instance['columns'][30]:
E               {'data': {'byteorder': 'little',
E                         'datatype': 'float32',
E                         'shape': [0],
E                         'source': 30},
E                'description': 'AB magnitude within the elliptical Kron aperture',
E                'name': 'forced_kron_abmag',
E                'unit': {'unit': '3.6307805e-20erg.Hz**-1.s**-1.cm**-2'}}

where the validation is now correctly marking this as an invalid column serialization.

[braingram]

This may also impact quantity:
https://github.com/asdf-format/asdf/actions/runs/21912149854/job/63268919595?pr=1999#step:10:927

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           asdf._jsonschema.exceptions.ValidationError: '1.4953789151759683e11' is not valid under any of the given schemas
E           
E           Failed validating 'anyOf' in schema['properties']['value']:
E               {'anyOf': [{'type': 'number'}, {'$ref': '../core/ndarray-1.0.0'}],
E                'description': 'A vector of one or more values\n'}
E           
E           On instance['value']:
E               '1.4953789151759683e11'

Somewhere a string is sneaking into a quantity.

[braingram]

Here is a relatively minimal file that incorrectly passes asdf validation:

#ASDF 1.0.0
#ASDF_STANDARD 1.6.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
t: !<tag:astropy.org:astropy/table/table-1.2.0>
  colnames: [kron_abmag]
  columns:
  - !table/column-1.1.0
    data: !core/ndarray-1.1.0
      data: [1, 2, 3]
    description: AB magnitude within the elliptical Kron aperture
    name: kron_abmag
    unit: !<tag:astropy.org:astropy/units/magunit-1.0.0> {unit: !unit/unit-1.0.0 3.6307805e-20erg.Hz**-1.s**-1.cm**-2}
  qtable: false
...

Changing table-1.2.0 to table-1.3.0 makes this correctly fail validation (since the tag validator used for checking the columns does not trigger the bug which results in a false passing of the validation).

If we strip out the table and just validate the column asdf validation correctly fails:

#ASDF 1.0.0
#ASDF_STANDARD 1.6.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
t: !table/column-1.1.0
  data: !core/ndarray-1.1.0
    data: [1, 2, 3]
  description: AB magnitude within the elliptical Kron aperture
  name: kron_abmag
  unit: !<tag:astropy.org:astropy/units/magunit-1.0.0> {unit: !unit/unit-1.0.0 3.6307805e-20erg.Hz**-1.s**-1.cm**-2}
...

[braingram]

This also impacts quantity (although in this case it's unclear if the python implementation can write out an invalid file).

[braingram]

At the moment I think the quantity issue is a bug in downstream (dkist) where the test file is invalid.

It's looking like the magunit issue can be handled in asdf-astropy (by making magunit behave more like a unit).

That should allow this validation issue to be fixed with minimal (ideally no) downstream impact.

[braingram]

This is worse than just visit_repeat_nodes. Here's a work-in-progress example:

import io

import asdf

yaml_str = b"""#ASDF 1.0.0
#ASDF_STANDARD 1.6.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
obj: !core/complex-1.0.0 foo
..."""

schema_id = "http://example.com/failing-1.0.0"
schema_str = f"""%YAML 1.1
---
$schema: "http://stsci.edu/schemas/yaml-schema/draft-01"
id: {schema_id}
properties:
  obj:
    oneOf:
      - $ref: "http://stsci.edu/schemas/asdf/core/complex-1.0.0"
      - true
..."""


with asdf.config_context() as cfg:
    cfg.add_resource_mapping({schema_id: schema_str})
    bio = io.BytesIO(yaml_str)
    with asdf.open(bio, custom_schema=schema_id, lazy_tree=True) as af:
        pass

This should fail validation since "foo" is not a valid serialization of "complex". However it passes on main and after enabling visit_repeat_nodes. This could just be a custom schema bug but in this case "obj" is never validated against the complex schema outside of the "oneOf" which is incorrect.