How to keep docs up to date

Problem: Documentation, by definition, creates a separate source of truth independent of the code. While we can autogenerate certain documentation like method signatures (e.g., godoc), there's no realistic automated system for documentation that broad areas like:

• key concepts
• system architecture (including diagrams, which are both expensive to maintain and highly valuable to readers)
• how and why to use certain {classes|methods|subsystems}
• how various pieces fit together

This issue is to collect ideas for what process(es) libp2p should put in place to stay up to date. I don't know what the best practice here is, but we should think about this early before it becomes an unmanageable nightmare.

Here are some ways I have seen this handled (none is perfect):

1. Minimize documentation - we should be careful about what we document. Especially diagrams, which are both expensive to change and extremely valuable to readers.

   • On diagrams specifically, we should look at having one (ideally open source) tool to create them and where we can keep the diagram "source" under version control and let people open PRs against it.
   • Another issue on diagrams is accessibility. We should try to have conventions around requiring alt text that explains what the diagram is intending to show. Unfortunately, this also has to be kept up to date.

2. Keep documentation isolated by topic - example: if the project adds support for a new NAT hole punching feature in 6 months, there should only be one article or article group that needs to be reviewed and updated.

3. Change log - each release[*] should have a change log. In different companies I've seen this done by engineering, product management, project management or documentation. I think the important point is just that we have some system by which one is being produced. Scripts can generate a list of all PRs between x and y git tags, but we need a human to group them together ("New: Feature X. This version adds support for feature X, which is [short description]. For more information, see [docs link] or these PRs [list...]")

   [*] Currently we don't even have a formal release process for every implementation, so this is an area that goes beyond just documentation.

   I've found that technical users like developers love change logs, especially ones that call out breaking changes. It softens the blow a lot when you change some behavior, even ones that were previously undocumented.

4. Versioned docs - this requires substantial extra infra work initially, but it's something to consider.

[yusefnapora]

Great points.

For diagrams, I've been looking at using mermaid, which is a text-based markup format for flow charts & interaction diagrams. It's not as flexible as something like dot / graphviz but is IMO simpler to read and write in its textual form. I haven't got very far with it yet, but hope to play with it some this week.

I think points 1 & 2 are both something to keep in mind as we structure the initial version - we should try to minimize dependencies between docs in the same way we'd think about module dependencies in code, if possible. One thing that could help would be for our "concept articles" to avoid discussion of implementation details as much as possible and instead lean on links to specs, etc, which change less frequently.

Things that seem especially likely to get out of date are the code-heavy tutorials we want to write, which will be frustrating for users if they don't track with code changes. We could look into having each tutorial be "backed" by a CI-testable example repo that gets triggered when its module dependencies publish new versions. Then we'd hopefully get an early warning if a release "breaks the docs". This requires a bit of setup and some discipline on the tutorial authors to make sure they're excerpting directly from the CI-tested example code.

Change logs are awesome - do we have any kind of script / tooling around this now?

By versioned docs, I'm assuming you mean docs that are for a specific libp2p release? That would be amazing. This ties into the formal release process you mentioned - if we can incorporate documentation updates into that process so we're confident that the docs are correct when we cut releases, we could keep a map of libp2p version numbers to IPFS links for each corresponding release of the docs site. In practice, we'd probably have separate "doc bundles" for each language implementation, with docs versions tracking that implementation's version number.

[salmad3]

Closing in favor of: #208.