typedoc Restructure

[shlomiassaf]

I have been using typedoc for a while, it has great potential and it works great but it's not developer friendly when it comes to plugins...

Here are some things I think would help:

• Allow ES6/ES2015-style import syntax for plugins (PR ready, support ES6/ES2015-style import syntax for plugins #594)
• Host library under an organisation (npm scope)
• Create monorepo structure for the repository, moving non-core plugins to a separate package under the org.
• Arrange extensibility related symbols in a single module (see info below)
• Simplify external plugin registration. (@Component wont work outside of the core package)
• Refactor JSON generation process to support a plugin architecture (PR Ready, Refactor JSON generation process to support a plugin architecture #597)
• Support object type for options (might come from js file, or CLI stringified argument). It will help complex plugins to "own" an object for options and not share the root options object.
• Think how to avoid auto-plugin registration (via @Component). This will allow inheriting plugins without registering them (or manually unreging them)
• Better docs... (cobbler, shoes, not there... )

The approach is to simplify the codebase, making the core small and allowing plugins to add features.

I have several plugins already built, thing like:

• 3rd party modules plugin: Make documentation aware of 3rd party modules either by linking symbols to external documentation or by fully embedding type info from d.ts files (2 modes)
  I was able to include the whole @angular docs, creating selective mashups etc... 2nd mode is not that useful but it's nice.

Supports binding import declaration (import X from 'Y') and namespace import declaration (import * as Y from 'Y')

• library-mode plugin: multi-mode plugin that allows exporting libraries from a single endpoint, creating a multi-lib documentation (monorepo) and other stuff...

• objectify: A plugin approach for creating JSON (toObject()). Reflection, Type, Source, Group, etc are all serialized to JSON via plugins that hook into events using the core plugin architecture. This plugin provides the core plugin logic and routing plus core toObject() plugin replacements for all native symbols (reflection, types, source etc..).

• json-datasource: Using objectify to tap in an convert the JSON into a datasource/table like structure, having all reflection in one reflection table (hash), types in type table, sources in source table etc... The graph is kept in tact by having pointers (id numbers) instead of objects.
  This is structure is suitable for SPA apps that require lookpups, quick access etc.
  In addition, the plugin will normalize the output so flags will output in their numeric form and not verbose form. It will add a table for each flag enum as a reference.

• json-links: Another extension to objectify that parses comments searching for markup links ([[ XX ]] or {@link ,,,}) and when found adds a link section that contains the link, caption, position and length so it can be replaced later.

• ng-decorator-meta: Create angular aware documentation. Uses 3rd party modules plugin for linking to angular docs and objectify to output the data to JSON.
  Comes with a group plugin that adds symbols to angular groups (Components, Directives, Output, Input etc...). It also hooks into the default theme to provide minimal UI representation.

Will post screen shorts in the next issue-comment

Arrange extensibility related symbols in a single module

Currently the entry point of the library exports about 6 modules (Application, CliApplication...). This is fine for simple usage through CLI or via API in node.
For plugin creators, importing is cluttered.. leading to this:

import { Component } from "typedoc/dist/lib/converter/components";
import { ChildableComponent } from "typedoc/dist/lib/utils";
import { ComponentClass } from "typedoc/dist/lib/utils/component";
import { Application, ProjectReflection } from "typedoc";

This is not developer friendly... making adoption harder.

An extensibility module should export all extensibility related symbols, this can be in typedoc/ext or @typedoc/core if moving to npm org/scope

Refactor JSON creation to support a plugins

The current method to convert all reflections/types/groups/sources/etc to one big JSON object uses inheritance, which does not allow customisation as a plugin system would. Since a plugin architecture already exists and the JSON convertion logic already exists (in toObject() methods) it is fairly easy to implement. I have a fully working solution the uses the current plugin system and just does convertion from Reflection, Type, Source, Group and any other type to JSON, just register a serializer and it will work.

Why? It will allow building SPA documentation app's that relay on a JSON or multiple JSON's.
Using my solution, I have added extra plugins to it that completely restructure the JSON making it more like a datasource/table like and less document oriented...

Note on CLI:

While using CLI exclusively is nice it does not scale. Once the complexity gets bigger, plugins start to pile and options/flags/customisability is required things become a nightmare to manage.
IMHO, using a config file (i.e. typedoc.js) is the best option and the one that should be recommended (and documented). This is where plugins should be configured, opt in-out etc.

[shlomiassaf]

Full metadata representation with type linking (both external and internal) and resource support (show html, css even of referenced files)

Decorators for members with external doc links:

[aciccarello]

Great work @shlomiassaf! Here are some of my thought. I'll let @blakeembrey chime in as he's got more control over the github organization.

Allow ES6/ES2015-style import syntax for plugins (PR ready, #594)

Thanks for the PR. That is a solid improvement and should ease some of the pain of people who are looking to create their own special plugin.

Host library under an organisation (npm scope)

This has worked well for Angular but I haven't used much beyond the CLI so I don't know how much this would help typedoc.

Create monorepo structure for the repository, moving non-core plugins to a separate package under the org.

👍 If we can structure it well. It should help with people looking for the theme and website code too.

Arrange extensibility related symbols in a single module (see info below)

I think that's a good idea, though not as high a priority. I'd prefer a full word rather than an ext directory name though.

Simplify external plugin registration. (@component wont work outside of the core package)

👍

Refactor JSON generation process to support a plugin architecture (full solution exists, see below)

That sounds like a good idea. Probably requires some design work.

Support object type for options (might come from js file, or CLI stringified argument). It will help complex plugins to "own" an object for options and not share the root options object.

Interesting idea. I'd be interested to see what this would look like. I would like to see better js file support/documentation too.

Think how to avoid auto-plugin registration (via @component). This will allow inheriting plugins without registering them (or manually unreging them)

Yeah, the decorator based design has some flaws that we need to work through.

Better docs... (cobbler, shoes, not there... )

😆 Yes please. Docs haven't been updated in awhile. Plus there is documentation scattered between the readme, different issues, and a few random places other than the website. We need to put some thought into what the strategy is.

[shlomiassaf]

@aciccarello

Host library under an organisation (npm scope)

This is well suited for typedoc, since the core typedoc engine is small and most of the work is done through plugins, it will allow people to actually choose what they want and for the project to be clearer, separated with less code clutter. typedoc, being a plugin oriented library is the perfect fit for scopes.

Arrange extensibility related symbols in a single module (see info below)

ext is just an example, once putting npm scopes in place it can be @typedoc/core or @typedoc/extensibility or whatever...

Refactor JSON generation process to support a plugin architecture (full solution exists, see below)

I have a pretty good implementation working, producing 1:1 output as the current source code.
I don't think a design is required since a design already exists, I just took the same design the other plugins implement (converters, themes etc...). Same thing, instead of in they do out... thats it.

[shlomiassaf]

@aciccarello see #597

[aciccarello]

It looks like the npm scope is taken so that doesn't look like an option. Probably will have to stick with a typedoc- prefix convention instead. I'd love to talk more about how to organize the extensibility symbols.

Beyond that I'm good with everything on the list. There is broken support for either a typedoc.json or typedoc.js file already implemented. It'd be great to get that fixed.

I've also put together a strategy for updating the docs. By migrating the site into this repo we should be able to update the API docs with a simple npm script. A plugins guide would be a great improvement too (see #521).

[mootari]

About the docs: I'm currently digging into TypeDoc and compiling some docs for myself along the way. I've created a gist containing what I have so far (component list, events, options per component): https://gist.github.com/mootari/d39895574c8deacc57d0fc04eb0d21ca

[shlomiassaf]

Guys I have access to typedoc doc I registered it so it won't be taken Don't worry about that I'll give it

…

Sent from my iPhone

On Dec 23, 2017, at 2:50 PM, Fabian Iwand ***@***.***> wrote: About the docs: I'm currently digging into TypeDoc and compiling some docs for myself along the way. I've created a gist containing what I have so far (component list, events, options per component): https://gist.github.com/mootari/d39895574c8deacc57d0fc04eb0d21ca — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[aciccarello]

@shlomiassaf 👏 Ah, I wondered if that was the case. (I assume you mean org). Great. That gives us some more options.

I'm hoping to make some progress over the Christmas holiday. Any help implementing/reviewing PRs is appreciated. Feel free to post questions and comments on gitter. I usually try to respond within a day. @mootari I appreciate the detective work you've done. ❤️ If all goes well I should be able to rebuild the docs in the next month.

[shlomiassaf]

I'm traveling this weekend so I'll not be able to do much... I'll post when I get back

…

Sent from my iPhone

On Dec 23, 2017, at 8:38 PM, Anthony Ciccarello ***@***.***> wrote: @shlomiassaf 👏 Ah, I wondered if that was the case. (I assume you mean org). Great. That gives us some more options. I'm hoping to make some progress over the Christmas holiday. Any help implementing/reviewing PRs is appreciated. Feel free to post questions and comments on gitter. I usually try to respond within a day. @mootari I appreciate the detective work you've done. ❤️ If all goes well I should be able to rebuild the docs in the next month. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[aciccarello]

I'm thinking for configuring a monorepo we should use lerna. Has anyone used that before? It looks like it would bring helpful conventions and easy linking/publishing. We could even use it for conventional commits.