Feature Request: F# style Type Provider support?

[battlebottle]

Could we get support for something like F#'s type providers in the future?

Let me present some use case scenarios for this.. Let's suppose you have a TS project trying to access a REST API, and this REST API uses Swagger or RAML for documentation. Well in this case we could have something like this;

var fooClient = new SwaggerClient<"http://foo.com/swagger.json">();
fooClient.getPeople((people) => {
    people.every((person) => console.log(person.firstName + "," + person.lastName);
});

and this could be completely type safe, as the SwaggerClient pulls down the API return types, method names (such as getPeople) and signatures at compile time and exposes them as methods to the type system. I think that's pretty huge. Such clients could be written by the community for RAML, SOAP or what have you.

another nice one:

var doc = new Document<"index.html">()
doc.titleSpan.text = "Hello World!";
doc.myButton.addListener("clicked", () => { console.log("tapped!")});

In this example the Document type provider loads our index.html file and finds all the elements with id's and figures out what types they are, and exposes them in the type system at compile time. Again, I think that's pretty huge!

F# uses Type Providers for all kinds of things, so there's lots of ways this can get used. It's easy to see frameworks figuring out all kinds of amazing things that these could be used for. It also feels like a nice fit for TypeScript because it helps bridge the benefits on dynamic typing with the benefits of static typing.

one more tiny example:

console.log(StringC<"name:%s/tid:%n">("Battlebottle", 128));

a simple string formatter. This reads "name:%s/tid:%n" and knows that %s and %n must be replaced with a string and number respectively, so at compile time it produces a method that accepts a string parameter and a number parameter. Basically a statically typed string formatter. Just a tiny of example of handy little utilities that could be written with this feature. But the opportunity is pretty huge I think in general.

Any thoughts?

[saschanaz]

👍 with the example for HTML. I don't like writing many declare vars to access them.

[mhegazy]

The idea is definitely something we have talked about on and off over the past few years. We will need a detailed proposal of how are providers declared, implemented, loaded and discovered.

[battlebottle]

I'm by no means any kind of expert on the TS compiler. I'm not even particularly advanced TS user right now. However, just to get the ball rolling I'll take a stab at a potential implementation for this.

There's obviously a bunch of different ways this could be done, and choosing the best way will depend on a deep understanding of TypeScripts goals as a language, how modern JS runtimes optimise code, among other things.

Here's a proposal for what the type provider might look like for the string formatter example:
(Please excuse the string formatting logic here, it's pretty much pseudo code. I didn't test and it's probably full of bugs.)

//Type Provider definition
StringF =
    @TypeProvider (str : String) => {//str eg: "id: %n/tname: %s"
        //Get array of format specifiers eg: ["s", "s", "n"]
        var formats = 
            str.split("%")
            .map((str) => {str.substring(0,1);})
        formats.splice(0, 1);

        //Convert formats to function arguments eg: [{name:"arg1", type: TP.baseTypes.String},{name:"arg2", type: TP.baseTypes.Number}]
        var funcParams = (() => {
            var types = formats.map((p) => {
                if (p === "s") {return TP.baseTypes.String;}
                else if (p === "n") {return TP.baseTypes.Number;}
                else if (p === "b") {return TP.baseTypes.Boolean;}
                else {throw p + " is not a valid string format specifier"}
            })
            var args = []
            for (var i = 0; i < types.length; i++) {
                args.push({
                    name: "arg" + i
                    type: type
                });
            }
            return args;
        })();

        return new TP.Function({
            parameters: funcParams //eg: [{name:"arg1", type: TP.baseTypes.String},{name:"arg2", type: TP.baseTypes.Number}]
            returnType: TP.baseTypes.String
            functionBody:
                @Expression """
                    var stringSplit = str.split("%");
                    for (var i = 1; i < stringSplit.length; i++) {
                        stringSplit[i] = stringSplit[i].substring(1, stringSplit[i].length - 1);
                    }

                    for(var i = arguments.length - 1; i >= 0; i--){
                        stringSplit.insert(arguments[i], i);
                    }
                    return stringSplit.join("");
                """
        })
    }

//Type Provider being used
console.log(StringF<"id: %n/tname: %s">(128, "battlebottle"))
console.log(StringF<"id: %n/tname: %s">(128, "ben brown"))
console.log(StringF<"id: %n/tname: %s/tage: %n">(128, "battlebottle", 30))

In this implementation, a type provider is simply a function that gets annotated with @TypeProvider. Doing this means that this method must be evaluated at compile time. The type provider function may accept parameters. In this example the type provider accepts a str : string parameter. When the type provider is referenced in code, it must be passed this argument as a generic parameter, such as in StringF<"id: %n/tname: %s">, "id: %n/tname: %s" is the str parameter. This parameter needs to be a literal as it must be evaluated at compile time. With this parameter, the type provider function will generate and return an AST node to represent the object, function etc that StringF<"id: %n/tname: %s"> will represent. In this case StringF<"id: %n/tname: %s"> will represent a function with the signature (arg1 : number, arg2 : string) : string.

As you can see, the TP.Function AST node constructor accepts an object with a parameters field, which must contain the name and type of all the parameters, a returnType field, which obviously declares the return type, and then a functionBody field. This one I'm less certain about, but here I currently have this as a string containing TS code representing the function body. Marking this string with @Expression could help let IDE's know that this string contains TS, and should perhaps be validated. This is something of a pragmatic solution. In F# the equivalent of this would be writing <@@ 1 + 1 @@> or what have you, with <@@ converting the expression inside into AST nodes. This is just syntactical sugar though for creating the AST nodes directly in code, and I don't think it does a whole lot for type safety. Evaluating JS string at run time is more normal in JS than doing something similar in C#/F#, and writing the function body as a string like this makes the whole process of creating a type provider a lot less complex than if the whole expression needed to be constructed with AST nodes. The TP.Function constructor defines the type signature of the function being created, which provides a layer of safety for when the type provider method is being evaluated, without making things feel as overwhelmingly complicated as writing F# type providers is today imo. I really have no idea what the best thing to do here. There's probably all kinds of things to consider here I haven't even thought about, but I'll leave this here as a starting point at least.

The type provider method is evaluated at compile time for each time the type provider object is referenced with different parameters. So for example:

console.log(StringF<"id: %n/tname: %s">(128, "battlebottle"))
console.log(StringF<"id: %n/tname: %s">(128, "ben brown"))
console.log(StringF<"id: %n/tname: %s/tage: %n">(128, "battlebottle", 30))

results in the type provider being evaluated twice, since two of these methods have the exact same parameter, they can refer to the same function.

The AST node that the type provider function returns, is converted to a real function (or anything else) at compile time for any code referencing the type provider, and these AST nodes are also used to generate the JS output.

So lastly, here is the JS that would be created from this code if we compiled it:

var StringF = function (str) {
    switch(str) {
        case "id: %n/tname: %s":
            return (function(arg1, arg2) {
                    var stringSplit = str.split("%");
                    for (var i = 1; i < stringSplit.length; i++) {
                        stringSplit[i] = stringSplit[i].substring(1, stringSplit[i].length - 1);
                    }

                    for(var i = arguments.length - 1; i >= 0; i--){
                        stringSplit.insert(arguments[i], i);
                    }   
                    return stringSplit.join("");
                });
        case "id: %n/tname: %s/tage: %n":
            return (function(arg1, arg2, arg3) {
                    var stringSplit = str.split("%");
                    for (var i = 1; i < stringSplit.length; i++) {
                        stringSplit[i] = stringSplit[i].substring(1, stringSplit[i].length - 1);
                    }

                    for(var i = arguments.length - 1; i >= 0; i--){
                        stringSplit.insert(arguments[i], i);
                    }
                    return stringSplit.join("");
                });
        default:
            throw "Could not find match for parameter: " + str;
    }
}

console.log(StringF("id: %n/tname: %s")(128, "battlebottle"));
console.log(StringF("id: %n/tname: %s")(128, "ben brown"));
console.log(StringF("id: %n/tname: %s/tage: %n")(128, "battlebottle", 30));

I've tried to keep to the TypeScript spirit of keeping things "just JavaScript", and making it so the JS output is still highly readable and could comfortably be used if the developer decided to drop TypeScript at some point.

As you can see StringF<"id: %n/tname: %s">(128, "battlebottle") is compiled to StringF("id: %n/tname: %s")(128, "battlebottle"). This keeps it looking largely familiar to the original TS code. The StringF variable, which was originally a type provider, is now a function. When StringF("id: %n/tname: %s") is called, it matches "id: %n/tname: %s" to the function that was generated at compile time for that parameter. Assuming the JS is unmodified, there will always be a function to match for the str parameter passed. I think this keeps things pretty readable, and if a developer wishes to drop TypeScript they could easily refactor this code to make it more idiomatic.

[Lenne231]

Type Providers for TypeScript would be awesome for something like Relay/GraphQL https://facebook.github.io/react/blog/2015/05/01/graphql-introduction.html
and FalcorJS
https://www.youtube.com/watch?v=z8UgDZ4rXBU

[Arnavion]

Another approach is to have a custom CompilerHost that generates foo-swagger.d.ts dynamically (preferably using some API like the one requested in #9147 as opposed to string concat), and either persists it to disk or services readFile requests for it from memory.