(auto-) generate a typescript `d.ts` file for better IDE Support

[jamestalmage]

Since basically every interesting method of this library is dynamically generated, it's impossible for IDE's like WebStorm to provide much help with autocompletion and/or static type analysis. I don't want to know how many hours I have lost context switching between my code and def/*, trying to figure out what the correct signature was for various build methods.

I created a minimal one by hand (currently only supports Identifier, ReturnStatement, and AssignmentExpression).

Note that it's utility would not be limited to Typescript users. IDE's can use them to provide hints for plain old javascript code as well. All the screenshots below are actually code hints generated for a javascript file.

Code Completion

Type Checking

Method Signature Hints

Webstorm just knows the type of the assign variable.

Everything we need to auto-generate the definition file is already in def/*, but we would probably need to extend Type to expose the information it contains in a format readily consumed by the generator (I guess I could parse Types toString output, but that seems unnecessarily difficult).

The generated file for the default language definitions shipped with ast-types should probably be deployed into npm as a static asset, but it would also be possible to generate it via a postinstall script. Ideally we would provide programmatic access as well, so those extending the language definitions (i.e. #57) could create their own definition files that reflect their modifications.

I am happy to submit a PR if @benjamn is on board. I am not sure how close #57 is to being a reality, but it would be nice to see what direction you were taking with that before starting on this in earnest.

[jamestalmage]

My hand generated file

// ast-types.d.ts
declare module AstTypes {

  export interface Type {
    check(x) : boolean
    assert(x) : boolean
  }

  export interface SourceLocationType {
    type: string
    start: PositionType
    end: PositionType
    source?: string
  }

  export interface PositionType {
    type: string
    line: number
    column: number
  }

  export interface PrintableType {
    loc: SourceLocationType
  }

  export interface CommentType extends PrintableType {
    value: string
    leading: boolean
    trailing: boolean
  }

  export interface NodeType extends PrintableType {
    type: string
    comments?: CommentType[]
  }

  export interface PatternType extends NodeType {}

  export interface ExpressionType extends NodeType, PatternType {}

  export interface AssignmentExpressionType extends ExpressionType {
    operator: string
    left: PatternType
    right: ExpressionType
  }

  export interface TypeType extends NodeType {}

  export interface TypeAnnotationType extends NodeType {
    typeAnnotation: TypeType
  }

  export interface IdentifierType extends NodeType, ExpressionType, PatternType {
    name: string
    typeAnnotation?: TypeAnnotationType
  }

  export interface StatementType extends Node {}

  export interface ReturnStatementType extends StatementType {
    argument?: ExpressionType
  }

  export interface NamedTypes {
    Expression: Type
  }

  export interface Builders {
    identifier(name: string): IdentifierType
    assignmentExpression(operator: string, left: PatternType, right: ExpressionType): AssignmentExpressionType
    returnStatement(argument?: ExpressionType): ReturnStatementType
  }

  export interface BuiltInTypes {
    string: Type
    function: Type
    array: Type
    object: Type
    RegExp: Type
    Date: Type
    number: Type
    boolean: Type
    null: Type
    undefined: Type
  }

  export interface Base {
    namedTypes: NamedTypes
    builders: Builders
    builtInTypes: BuiltInTypes
  }
}

[benjamn]

This is a really neat idea. I'm happy to make any adjustments necessary to ease the auto-generation of the interface file.

What if Types had a .toDeclaration() method that generated the export interface ... text, similar in spirit to .toString()?

Once https://github.com/estree/estree is more complete, I'd like to consume it as an input format, so it would be great if ast-types can play a larger role in generating different kinds of APIs and output formats.

[jamestalmage]

What if Types had a .toDeclaration() method that generated the export interface ... text, similar in spirit to .toString()?

Yep, I was thinking along those lines.

It might also be necessary/desirable to add an optional parameter to the Type constructor that allows users to create custom output. Similar to how the name parameter is currently used. I may be getting ahead of myself here.

Also, it seems defineMethod would definitely benefit from having the option to specify the method signature.

Once https://github.com/estree/estree is more complete, I'd like to consume it as an input format, so it would be great if ast-types can play a larger role in generating different kinds of APIs and output formats.

Interesting. Thanks for the link. Is anyone from the Typescript team involved with that? The syntax looks pretty similar.

[jamestalmage]

I am going to collect some relevant resources in this comment. Just for my reference as I work on this:

• best practices - codeplex
• best practices - definitelytyped
• The definitelytyped project creates tests that are actually just typescript files that should compile without error. Probably a good idea for validating our output. (example) (mentioned in the contrib-guide)
• Instructions for referencing generated file in package.json for automatic detection and easy consumption via the tsd command line tool
• Some of the previous links have outdated information, especially in regards to union types. (implementing PR)

[jamestalmage]

Maybe not the final solution, but this could also help with #44.
It would require generating an interface for the object passed to types.visit. It would just define a bunch of visitXXX methods (all of them optional). Added benefit is that you would get code completion in there as well.

interface NodePath<T> {
  node: T
  // ...
}

interface Visitor {
  visitReturnStatement?(path:NodePath<ReturnStatementType>):any
  // ...
}

[jamestalmage]

See Declaration merging possibly applicable if we process each file in def/* separately instead of as a whole. Doing so would allow us to create separate d.ts files that cleanly represent how each module expands the type definitions (for example def/fb-harmony adds a typeAnnotation field to Identifier).

Separate, d.ts files that cleanly extend a core.d.ts file would be desirable in that users could pick and choose which language features were available without programmatically compiling their own. Also, it feels like the "right way" vs generating a single monolithic file for distribution. That said - doing it correctly may be tricky, especially before #57 is complete. Might be more trouble than it is worth.

[jamestalmage]

@benjamn
It seems somebody from the estree team has already generated some of these. Should we just generate our own, or base of those somehow?

[RReverser]

It seems somebody from the estree team has already generated some of these. Should we just generate our own, or base of those somehow?

Yeah, I just generated spec-compliant definitions and updating from time to time using a script.

[KazimirPodolski]

Here is generated declarations for core part: https://gist.github.com/spontaliku-softaria/056a6c641b89a95a2afaa10cccdea944.

Edit: I understand these are incomplete and unfinished, but it allows IDE completions right now for core functionality. I believe that to do proper generation a refactoring is needed. In particular, there should be a way to get correct type names for any field definition. Currently things like isNumber + " >= " + than and Identifier | null leak from field definitions to the model. They are unreadable for external processing (like declarations generator). Personally I think type names should not be an interface for Types (because string Identifier | null is not a valid type).

[cancerberoSgx]

@spontaliku-softaria hey, do you plan to PR this work ? I think having .d.ts will add great value, thanks!

[KazimirPodolski]

@cancerberoSgx no, I'm not using the tool where I needed declarations anymore. And complete declarations might requre big changes in ast-types, which I'm not qualified to do.

[bnjmnt4n]

This issue can be closed since #300 was brought in.