Proposal: Bundling TS module type definitions

[weswigham]

Relates to #3159, #4068, #2568, this branch, and this tool.

Goals

• Bundle declarations for TS projects to allow a library to be consumed with a single TS file, despite being many modules internally. Because of complicated internal module dependencies which would rather not be exposed to the consumer, this should flatten the exported module types as best as it can. (Ideally, completely.)

Proposal

When all of --module, --out, and --declarations are specified, the TS compiler should emit a single amalgamated .d.ts (alongside its single output js file). This .d.ts should be flattened compared to a concatenated .d.ts file. It should report collisions caused by scoping issues and import aliasing when flattening declarations into a single declare module. It should respect access modifiers when generating the DTS (only exporting things explicitly exported and types marked as public).

For example, given the following set of sources:
tsconfig.json:

{
  "compilerOptions": {
    "module": "commonjs",
    "declarations": true,
    "out": "mylib.js"
  }
}

a.ts:

export * from './b';
export * from './c';

b.ts:

export interface Foo {}

export class Bar {
    constructor() {
        console.log('');
    }

    do(): Foo { throw new Error('Not implemented.'); }
}

c.ts:

export class Baz {}

should create the .d.ts:
mylib.d.ts:

declare module "mylib" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }

  export class Baz {}
}

rather than:
mylib.d.ts:

declare module "mylib/a" {
  export * from "mylib/b";
  export * from "mylib/c";
}
declare module "mylib/b" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }
}
declare module "mylib/c" {
  export class Baz {}
}
declare module "mylib" {
  export * from "mylib/a";
}

and should report a semantic error when the following is done:
a.ts:

export * from './b';
export {Bar as Foo} from './b';
export * from './c';

as there will be multiple members named Foo (an interface and a class), since b.ts has exported interface Foo.

We should also have a semantic error when the following is changed from the original:
If we change c.ts:

export class Baz {}
export interface Foo {}

it should be an error in a.ts (since it's blanket exporting b and c), and the error should suggest to alias either c.ts's Foo or b.ts's Foo (or both) when reexporting them in a.

Internally, when flattening this aliasing becomes important - we need to track usages of the two original Foo's across the generated .d.ts and rename it to the alias created when it is reexported.

Unfortunately, to maintain ES6 compatability, while we can warn about this behavior with classes (since it's possible that a developer is unaware they're overriding a prior export), we still need to support it (or do we? The spec leads me to believe that attempting to export multiple members with the same name - even via export * - is an early syntax error). So it would be nice to have a compiler flag to mark the same kind of thing with classes (or namespaces) as an error, but also do the following by default:

We can do automatic name collision resolution, but that can result in unpredictable (or convention-based) public member names... but it must be done, I suppose. We could ignore reexported types since it's appropriate to do so in ES6 (following export * declarations can override previously defined members? maybe? system works this way at present - but that may just be system relying on transpiler implementers to maintain ES6 semantics), then we would need to create "shadowed" types at the appropriate level in the .d.ts - types whose original public access are overridden by later exports but whose types are still required to describe public function argument or return types. Naming these "shadowed" types could be difficult, but given that they only exist for type information and not for access information, a common (re)naming convention could be a desirable solution. Something akin to <typename>_n when n is the shadowed type number for that type, and renaming the shadowed type name to something else (<typename>__n and so on so long as the name still exists) if that collides with another exported type. Classes used in this way are rewritten to interfaces in the .d.ts, since a constructor function likely isn't accessible for a shadowed class (at least not at its generated exported type name).

Any feedback? There's a few alternatives to what I've suggested here, which is possibly the most conservative approach in terms of ability to error early but supporting ES6 semantics best. It's possible to silently ignore interface name collisions and rename those automatically as well, but since they're TS constructs and not ES6, I think it's okay to force more discipline in their usage.

Something I've been considering is also rewriting namespaces as interfaces in the generated .d.ts in this way to further flatten/unify the types, but this... might? not strictly be needed. I haven't come up with a strong case for it.

[weswigham]

I realize that I've forgotten to propose a way to define the entrypoint module (as is possible in #4434), and I suppose that warrants discussion, too.

The entrypoint is the location which (in the above example) TS considers the 'top level' for flattening types (in this example, a.ts). Ideally, this is the entrypoint to your library or application. All of the dependent files in a well-formed module-based TS project should usually be accessible from the entrypoint via either imports or triple-slash refs... however In TS's default mode of operation for the --module flag, TS ignores most relationships between included files and compiles them mostly separately, resulting in separate js and dts files for each. Like the proposal in #4434, dts bundling may also make sense to require a --bundleDefinitions [entrypoint] flag, like how bundling module sources could require the --bundle [entrypoint] flag.

On the other hand, rather than add a new compiler flag, we could consider all files specified as 'root' files as entrypoints when compiling with --module and output a definition file for each of them. (Meaning that all other files need to be accessed by those root files and compiled via that relationship.) Conceptually, we could do the same thing with #4434, rather than having the --bundle argument to specify an entrypoint. This does lose the meaning of the --outFile argument, however, since there are suddenly multiple output files (one for each root file) and none of them correlate to the --outFile parameter... So maybe it's best to not try to rely on traversing the dependencies ourselves and require an extra parameter specifying the entrypoint.

[NoelAbrahams]

@weswigham,

I'm generally in favour of this proposal. I think this feature would help to formalise the idea of a project in Visual Studio (i.e. types defined in namespaces compiled into a a single output file, that can then be referenced by other projects).

One of the problems with a single output file is that it makes life difficult when debugging in the browser. See #192 (comment). Ideally something can be worked out for that as well.

[weswigham]

#3159 implements most of this proposal, though it restricts it to ES6 or commonjs modules, uses slightly different terminology for tsc command line flags. and omits the type flattening aspect of this proposal. At the very least, it's probably an excellent starting point for talking about this.

[alexeagle]

some notes from doing this for Angular:

• should also report a semantic error when a symbol is re-exported to an entry point, but a dependent symbol is not (for example, declared supertypes, parameter types)
• we like to emit a namespace as well as a module, allowing users to use symbols from that global namespace without any import statements (eg. useful for ES5 users in VSCode to get intellisense).
• we might want to be able to use this typings bundle feature without having to use the runtime emit bundling feature from Proposal: Bundling TS modules for consumption and execution #4434 - for example if there are some errors producing a working emit because of private APIs, it would be nice if we could still use this to produce the public API doc.
• We like to preserve some comments so that tools can show inline doc when showing eg. a completion tooltip. Should probably explicitly mention comment handling in the proposal.

[alexeagle]

We can't drop our current .d.ts bundler without constructor visibility: #2341