Custom Tag Implementation

The class AsdfType is the parent of all tag class implementations. This includes all of asdf’s core types, such as NDArrayType and TableType. It is also the parent of TransformType, which is relevant to our current discussion. Critically, any third-party software that implements a custom tag class will use AsdfType. It says as much in the documentation for creating custom extensions.

However, there’s a problem. The metaclass constructor for AsdfType adds every single child class of AsdfType to the list of asdf’s built-in tag types. These all get bundled into asdf’s built-in extension, which is what asdf uses to locate the associated schema files. To provide an example, pretend I have a paleontology software package that uses asdf to store dinosaur data. If I create a custom tag class called TheropodMetatarsalType, and it inherits AsdfType just like the documentation says it should, then asdf is going to automatically and silently add TheropodMetatarsalType to its list of built-in types.

I want to convince you that this is not okay. I also want to convince you that even if it was okay, it’s still broken.

First of all, it’s not necessary. Asdf provides a mechanism for handling custom extensions (namely through the extension keyword argument to asdf.open and the AsdfFile constructor). This is how third-party software should inform asdf about custom types.

Second of all, it breaks things. It makes the asdf module itself stateful. To make a long story short, if you have multiple unit tests that define different versions of the same tag class, it’s not going to work since each of those gets added to a global registry of asdf built-ins. There are reasons that this is would not be desirable in production code as well. Maybe my program wants to process multiple asdf files that all use different sets of custom types. It doesn’t make sense for all of those custom types to pollute the module’s common list of built-ins.

Third of all, it doesn’t work. Even though my custom TheropodMetatarsalType gets added to the asdf built-in extension, it gets added with the wrong path to the schema file. So, my code won’t work unless I also happen to define a custom extension containing my new type, and explicitly use it when I open my asdf file. Yuck.

So how do we solve this problem? My solution was to create a UserType that does not implicitly add its children to the list of built-ins. All third-party tag classes should inherit this instead of AsdfType. We can argue about names (e.g. maybe this new class should be called AsdfType and the built-in only one should be changed to something else), but I think something like this is the right solution.

The question is what do we do about gwcs types that inherit TransformType? Should these be considered third-party extensions and so should really inherit something like UserTransformType (which doesn’t yet exist)? Or should they be allowed to be treated as built-ins? If the answer is the latter, then see the comment above about the extension referencing the wrong schema file path. This will need to be fixed, and I suspect fixing it is going to be fairly ugly.