Support looking for modules under node_modules when importing

[chanon]

Update 5 November 2015

The functionality requested below is currently implemented in typescript since at least 1.8 with one main difference:

Instead of having typescript.main and typescript.definition properties, there is only one typings property which you can point to either a d.ts file or a normal .ts file.

If you're developing a module to just use locally, you can have the typings point to a .ts file, but if you plan to publish the module, it is recommended to have it point to a d.ts file. This is because you don't want your module consumers to recompile your module files, just consume its typings.

I have setup an example of using this here:
https://github.com/chanon/typescript_module_example

There is a documentation page here that has more information:
http://www.typescriptlang.org/docs/handbook/typings-for-npm-packages.html

Thank you TypeScript devs and all contributors.

Original issue / feature request follows

---

Motivation

In TypeScript it is a lot harder to re-use typescript modules compared to re-using npm modules in JavaScript.

It would be beneficial if the typescript compiler is smart enough to look in node_modules folders and package.json files.

The reason is so that npm module developers that use TypeScript might be able to start writing and distributing modules through npm itself. TypeScript would be able to piggyback on npm's infrastructure and wide support.

Example for node_modules

If we had:

./node_modules/concator/index.ts
./myApp.ts

And in index.ts we had:

export function concat(param1: string, param2:string): string {
      return param1 + ' ' + param2;
}

in myApp.ts:

import concator = require('concator');  // loads the module from node_modules
var result = concator.concat('I like', 'this.');
var wontWork = concator.concat('this will fail');  // compile error, concat needs 2 params

So basically, the compiler is smart enough to find the module in node_modules and it automatically uses the typescript version (index.ts).

Then when the code is compiled to JavaScript, it naturally uses the JavaScript version.

Importing Folders as Modules

A more basic case is supporting Node.js's popular rule of http://nodejs.org/api/modules.html#modules_folders_as_modules as suggested by @vvakame below and in the closed (semi-duplicate) #207 issue.

typescript.main in package.json

There could be an addition to package.json files to specify where the main .ts file of a TypeScript npm module is. This would be similar to the main key that specifies where the main JavaScript / .js file is but for TypeScript instead.

Eg package.json for an npm module named "myModule" located at node_modules/myModule/package.json

{
     "main": "./dist/index.js",
     "typescript": {
          "main": "./src/index.ts"
     }
}

In this example node_modules/myModule/src/index.ts would be the main TypeScript file and doing an import myModule = require("myModule"); would import the node_modules/myModule/src/index.ts file.

For a JavaScript coder writing var myModule = require("myModule"); in a JavaScript file, the require would load the node_modules/myModule/dist/index.js file as usual.

As can be seen in this example, the TypeScript src is in the node_modules/module-name/src folder and the compiled JS files would be in node_modules/module-name/dist.

Something like this could be a (semi) standard for TypeScript npm modules so that the TypeScript source is cleanly separated from the compiled JavaScript output.

A more basic case as suggested by @vvakame below is supporting the popular Node.js rule of http://nodejs.org/api/modules.html#modules_folders_as_module

typescript.definition in package.json

Another possible key for package.json for non-TypeScript (plain JavaScript) npm modules could be typescript.definition. This would point to a .d.ts file that defines the module for TypeScript users of the npm module.

So that an
import $ = require('jquery');
would automatically read a jquery.d.ts file defined in the typescript.definition key in jQuery's package.json and make $ the correct type.

Example format:

{
     "main": "./dist/index.js",
     "typescript": {
          "definition": "./index.d.ts"
     }
}

(This format is already used by tsd as @Bartvds explains below.)

Then us TypeScript coders would just have to try to get as many non-TypeScript npm module maintainers to merge our pull requests that have our .d.ts files and package.json typescript.definition keys.

If we succeed with that, then TypeScript coders' life would be bliss ... no more separate managing of DefinitelyTyped .d.ts files. Just npm install and you get your TypeScript definitions too! Automatically and up-to-date with the module version installed.

List of Benefits

What we get from all of this is

• npm modules can be written in TypeScript.
• Both TypeScript and JavaScript coders can use these modules
• The module users who use TypeScript get the benefits of static typing without the module coder (or user) having to use the usual method of writing (or generating) separate .d.ts files (so when the sourcecode of the module is already in TypeScript we don't have to have another .d.ts file to maintain)
• There would be a way to easily re-use and distribute TypeScript modules using npm which everyone is already familiar with
• It could help promote TypeScript usage itself, as JavaScript coders who use the npm modules written in TypeScript might see how nice / better the TypeScript source is and try switching to it. Or they might want to contribute to the module and since the source is in TypeScript they would have to learn it which may lead to them liking it and deciding to use it in their own projects.
• Basically it would help grow the TypeScript community through all the code sharing that could result. Right now everyone's TypeScript code is mostly their own / in their own silo. This is actually probably an extremely important thing as not having a proper/easy way to share/distribute/re-use modules is probably limiting the growth of the language. [1]
• Re-use of modules in internal projects would be a lot less of a hassle and would reduce the need for all those relative paths to .d.ts files and relative paths in module requires, and general .d.ts stuff. (Right now when writing TypeScript code, I find I have to learn to be good with counting ../../ s)
• This approach also supports Browserify/Webpack as Browserify/Webpack also look under node_modules, so this allows for easy reusable / npm distributed TypeScript modules for both the server and browser
• Non-TypeScript npm modules can easily be used in TypeScript with type information through the typescript.definition key addition. This allows .d.ts definition files to be packaged together with the npm module so that updating an npm module will automatically update its .d.ts definition file. This removes the need to update .d.ts files manually.
• Using definition files through the typescript.definition is simpler because it is simply an import moduleName = require("moduleName") statement with no need for a separate ///<reference ...
• Using definition files through the typescript.definition should also allow the use of different versions of the module in the same code base without type names clashing.

Detailed Proposal

@Nemo157 has written a very detailed proposal for how all of this should work at:

Proposed TypeScript Require Resolution Semantics
https://gist.github.com/Nemo157/f20064a282ee620f3877

An addition in the proposal is the usage of /typings folders which can hold definition files that can be automatically managed by tools such as tsd for JavaScript npm modules that won't include definition files in their repositories.

Final Supporting Facts

Since TypeScript compiles to JavaScript which runs mainly in two places: node.js and in browsers, supporting node_modules is beneficial to both places (practically all of where TypeScript is used) because of npm and Browserify/Webpack.

ie. there is no reason to come up with a different scheme when node_modules is what all TypeScript users use already for maybe 75%-100% of all their JavaScript code. (Taking out 25% for maybe RequireJS users.)

Footnotes

[1] - BTW I see there is a NuGet package manager (?) from Microsoft that can distribute typescript modules (?), but coming from a node.js focused (non .NET focused) background, I don't see NuGet becoming widely used outside of Microsoft focused shops, especially as npm is the standard for node.js and is a leading standard for client side JavaScript too. If I didn't use TypeScript I would have never heard of NuGet. The average node.js / client side JavaScript coder would probably prefer using npm, the same tool they use already rather than having to use something Microsoft specific such as NuGet. (I don't actually know a thing about NuGet so what I'm saying here might not actually matter.)

[chanon]

[moved to typescript.main in package.json in the issue description above]

[vvakame]

👍
I think http://nodejs.org/api/modules.html#modules_folders_as_modules is very popular rule of Node.js.

other example.

./test/index.ts

export function hello() { return "Hello, world"; }

./main.ts

import test = require("./test/");
console.log(test.hello()); // print "Hello, world"

[chanon]

[moved to typescript.definition in package.json in the issue description above]

[basarat]

One solution would be to solve it with better tooling. E.g. import foo = require('foo') gives you a hint to look for some local node_module+package.json (foo is a ts project) or DT definition (foo is a js project where library authors don't want to maintain the ts def).

[Bartvds]

FYI; I'm actually testing something similar to this in TSD; a way to expose & link definitions that are bundled in the npm (or bower) packages.

It's in my dev version for 0.6, and it works by adding a typescript element to the package.json (or bower.json), with a sub element definition (a sub element because maybe one day there'd be source too, or whatever).

{
    ...
    "main": "./index.js",
    "typescript": {
        "definition": "./foo.d.ts"
    }
    ...
},

Then you can run a command on TSD, currently tsd link and it will scan all package.json files in node_modules (or bower or whatever), find that property if defined and add a reference to it to the central tsd.d.ts bundle in your project.

Example: defined here and use here

[chanon]

@Bartvds That's pretty nice. Could be the first step for having .d.ts in npm packages. I like the package.json structure with "definition" in "typescript", it's a lot more organized.

If TypeScript compiler itself could read it automatically, would be very cool.

[RyanCavanaugh]

Tagging this for discussion -- smarter external module resolution is something we need to talk about to understand the various scenarios here. This was brought up previously and we made a few tweaks in 1.0.

[RyanCavanaugh]

Also discuss #207 - looking under 'index'

[Evgenus]

👍

@chanon tsMain is not necessary if we use declarations generation.