Core feature: Server Metadata

[warriordog]

Background

ActivityPubSharp is designed to work with many different AP applications, even those that aren't spec-compliant (like fediverse sofware). This has historically been a huge pain point that every AP project must address. While many varied solutions exist, we are taking an entirely different approach from any other known implementation.

Overview

APS will implement a concept called "Server Metadata", which represents auto-detected metadata about a particular domain. Server Metadata, or server meta, will contain information like this:

• Type of software running on a particular domain
• Compatibility flags such as:
  • Extensions? Which ones?
  • JSON-LD?
  • Will it blow up if type is an array? // This one is critical for Mastodon support
• Delivery records / statistics
• Known actors

This information will be made useful by a process known as rewriting. The rewrite process involves middleware that analyzes an object intended for delivery and compares it against the recipient's server meta. All fields and data in the object are checked, and anything incompatible is "rewritten" to a supported format. The actual changes made will vary based on the type and version of software running on the target domain.

Challenges

Obviously, this is not simple to do. There are concerns to keep in mind and many challenges to overcome:

• Metadata must be detected transparently and *automatically.
• Metadata must be used transparently and automatically.
• Collecting accurate meta will require the use of related protocols, like NodeInfo.
• All target software must be supported in advance. This is a forward-compatibility risk and a tremendous development effort.
• To mitigate the previous risk, this process must be extensible. It should be possible for an application to plug in a custom converter for a new / updated remote application, without needing to update or modify APS itself.
• Not all features can be rewritten. There must be a mechanism for upstream code to determine whether a fully-incompatible message is dropped, truncated (remove the incompatible parts), or errors out entirely.
• Given the risks of this feature, it must remain optional without much duplicated code or configuration.
• We won't be able to support every possible application or version. We will need some kind of fallback, probably "rewrite to strictest spec compliance" or maybe "emit as-is". Application code should be able to select or even replace the fallback logic.

This is not an easy or trivial project. We need extensive research and design, and probably a prototype phase too. We should also ensure that the Type Model and Use Case architecture are nailed down first.

Expectations and Assumptions

This concept and design make several assumptions about existing and future AP implementations. While it's true that every guess is a potential bug, there's unfortunately no other option for this project. The best we can do is recognize and document all expectations.

• A particular application will never lose support for a feature. If it does, then any compatibility problems are the fault of it's creators. We will of course address these with a new app-version pair, but existing APS deployments will remain broken. We don't consider that a bug or failing, as the exact same thing would occur between old and new versions of the application in question.
• Compatibility problems can be addressed at the protocol level, with hinting from upstream code.
• All outgoing objects contain the greatest superset of all potentially-needed data. Converters cannot obtain new information - only cut out or reorganize what's already there. Application code should stage the broadest-possible object to cover everything that they wish to support.
• Our type model is flexible enough to both support the rewriting and contain a superset of information.
• All remote software can be detected in some way.
• Remote servers do not lie about their implementation.

Example Flow (very old and incomplete)