[require()] installing from npm registry, requiring, uninstalling, then installing a different version then requiring again, attempts to execute the old main script

[akhoury]

node v0.10.3, npm v1.4.4

even though package.json was clearly updated on the file system.

If the old main script does not exists anymore the require() call fails, even if require's cache was cleared. I am not sure if that's an npm or nodejs bug.

I've written a simple gist (if you would like to run it), of just checkout the logs output as a comment on the gist, and filed the same bug at the npm repo
https://gist.github.com/akhoury/c6104082c06749895535

thanks

[indutny]

I think that npm repo is a right place for this. Thanks for sharing, though.

[akhoury]

I couldn't re-open the bug but I'd like to add this.

I highly suspect it's a node bug, with Module._findPath(), specifically that the main file cache-lookup-key does not include version or the origin of the module, just the request-id and the local paths stringified.

After further debugging, it seems all scripts paths are cached in Module._pathCache
https://github.com/joyent/node/blob/master/lib/module.js#L163

[akhoury]

        Object.keys(module.constructor._pathCache).forEach(function(cacheKey) {
            if (cacheKey.indexOf(moduleName) > 0) {
                delete module.constructor._pathCache[cacheKey];
            }
        });

that fixed it.

or more like

/// inspired from http://stackoverflow.com/a/14801711/493756

   var searchModulesCache = function(moduleName, callback) {
        var mod = require.resolve(moduleName);
        if (mod && ((mod = require.cache[mod]) !== undefined)) {
            (function run(mod) {
                mod.children.forEach(function (child) {
                    run(child);
                });
                callback(mod);
            })(mod);
        }
    };

    var reloadModule = function(moduleName) {

        searchModulesCache(moduleName, function(mod) {
            delete require.cache[mod.id];
        });

         // extra cautious and delete the Module._pathCache entry just in case.
        Object.keys(module.constructor._pathCache).forEach(function(cacheKey) {
            if (cacheKey.indexOf(moduleName) > 0) {
                delete module.constructor._pathCache[cacheKey];
            }
        });

        return require(moduleName);
    };

/// example usage: 
var myModule = reloadModule('my-module');

[akhoury]

I am not sure if that's a bug or works-as-designed anymore, but the Module._pathCache caching could potentially be more specific and include the module's version if possible.

Thanks.

[sbellone]

Hello. Will this issue be reopened since we now know that it comes from node?

I've tried @akhoury workaround but it's not enough: the main script file is kept in cache in the packageMainCache property (which is not accessible from outside).

So for example if I update winston from a 0.6.x version to 0.7.x version (where its dependency to async is updated to 0.1 to 0.2), and reload the winston module, I am getting a Cannot find module 'async' error.

Maybe we could at least expose the packageMainCache property?
Thanks.

[akhoury]

@sbellone good catch. could you write up a quick test.js that shows your case ?

[sbellone]

Sure, here it is: https://gist.github.com/sbellone/3947632a42be73553abb

[akhoury]

see my fork. I think I have a very ugly workaround.

[akhoury]

I think the best solution is to have all cacheKeys include full path and module exact versions in them, that needs to be changed in Node's core; Module.

[akhoury]

@indutny this issue end up being in node core. Could this issue be re-opened or at least investigated?

Module._pathCache and var packageMainCache do not play well when you switch between different versions of the same module.

I suggest that the version (or the compressed package-url) should be included in the cache key

[thelonglqd]

Any updates about this issue, I stuck in here for 2 days long :(

[julianlam]

Yeah, pinging @indutny so he can re-evaluate whether closing was the right course of action here 😄