Files changed by tasks stared by watch with nospawn on do not trigger new events

[iclanzan]

If we take this example from the docs, which runs fine:

grunt.initConfig({
  sass: {
    dev: {
      src: ['src/sass/*.sass'],
      dest: 'dest/css/index.css',
    },
  },
  watch: {
    sass: {
      // We watch and compile sass files as normal but don't live reload here
      files: ['src/sass/*.sass'],
      tasks: ['sass'],
    },
    livereload: {
      // Here we watch the files the sass task will compile to
      // These files are sent to the live reload server after sass compiles to them
      options: { livereload: true },
      files: ['dest/**/*'],
    },
  },
});

and add the nospawn: true option, the livereload target will never be called even though sass changes files.

Any thoughts on this?

[shama]

I'll take a look... but why do you need spawn: false here?

[iclanzan]

For performance reasons mainly. With spawn: true I have to wait in excess of 2 seconds for the reload to happen after a change. Without spawning a new process it takes 0.02 seconds. That’s 100 times faster.

[tillda]

I can confirm this. With spawn:false somehow if "livereload" task watches a location being changed by another task then the change doesn't get noticed by "livereload".

[tillda]

Actually, I have found a pretty neat workaround: start another grunt process with a specific --gruntfile that handles only LiveReload. Since I compile everyhing to pure-html&css&js app in a separate /build directory I don't need a lot of LR configuration. Also it solves the problem of grunt-LR tasks waiting for others to finish, because in the other process the filesystem change is noticed immediately, IMHO.

[iclanzan]

@tillda I love your workaround.

[codepunkt]

I'm having the same problem:

• sass files in 'src' directory are compiled to .css files in 'build' directory
• additionally, .css files in 'build' directory are watched to trigger livereload

watch:
    options:
        livereload: false
        # spawn: false
    build_files:
        files: [ '<%= build_dir %>/**/*.css' ]
        options:
            livereload: true
    sass_files:
        files: [ 'src/**/*.{sass,scss}' ]
        tasks: [ 'compass' ]
    source_files:
        files: [ 'src/**/*.{sass,scss}' ]
        tasks: [ 'copy:source_files' ]

Example is shortened, but shows the problem.
Using spawn: false results in acceptable speeds, but doesnt trigger livereload.

@shama
Any idea whats causing this?

@tillda @iclanzan
Is it possible to start that new grunt process from the first grunt process?

[tillda]

@gonsfx I don't know. I made a tool for managing a development processes that run in background: https://github.com/tillda/pm.js

[codepunkt]

In my example, the livereload of the compiled .css is triggered on every second save of the .sass file.
Looks like the taskrun of the build_files task is only completed on every second run.

@shama Seems i can work around this by setting the interval option to 1 - which seems awkward. Does gaze use fs.watchFile (requiring an interval) instead of fs.watch?

[shama]

This happens because Grunt only runs one task at a time. With spawn: false, the watch task must queue tasks, queue itself to run at the end of those tasks then stop itself to run the queue. While the queue is running, it currently doesn't watch for changes (but the default spawn: true will). FWIW, I'm working on a solution for this.

Gaze uses a combination of fs.watch and fs.watchFile. To delay hitting EMFILE errors sooner, node v0.8 compatibility and to handle safe rewrites among other use cases. fs.watch gets a bit better in newer versions of node and I'm looking into ways we can rely on it more if you're using node > 0.8.

So if setting interval to 1 works for you, that's fine. It would likely heat up the CPU for users with larger file sets though. So just be careful with such a low interval.

[codepunkt]

Its not a solution i'd really like to use, was just pointing it out in case it might help solving the problem.

I was digging around in the code of this and gaze for a few hours today, but i neither found the exact reason for this behavior nor a feasible solution to fix it.

After reading your latest comment - shouldn't it be possible to run the queue and the watch task itself in two different processes?

If i can somehow help you out with this, drop me a note.

[shama]

That is what spawn: true or the default does. The watch stays in one process and spawns task runs in another. It's a good default for most users as the watch will stay alive even if the tasks it runs crashes. But it's about ~500ms slower to spawn up a new process and doesn't share the process context (so whatever you change in your Gruntfile when it's running doesnt get changed in a spawned run).

I'm experimenting now with preemptively forking a grunt process that waits to be triggered: https://github.com/shama/wait-grunt This should speed up the perceived spawn time but doesn't help much with the context issue. As we only get a channel to send data between processes.

[princed]

Complete duct tape solution:

module.exports = function (grunt) {
  require('load-grunt-tasks')(grunt);

  var WATCH_TASK = 'watch';
  var watchConfig;

  if (process.argv.indexOf(WATCH_TASK) === -1) {
    grunt.util.spawn({
      grunt: true,
      args: [WATCH_TASK]
    });

    watchConfig = {
      options: {
        livereload: false,
        spawn: false
      },
      // tasks to produce target files 
    };
  } else {
    watchConfig = {
      options: {
        spawn: false
      },
      livereload: {
        files: [
          // target files to watch
        ]
      }
    };
  }

  grunt.initConfig({
      watch: watchConfig
      // other tasks
  });
}

[mnishihan]

@princed 👍 for your solution. But this does not work when there are multiple watch tasks to be run in parallel along with some other blocking tasks like compass:watch using grunt-concurrent. After trying for a while, I could figure out a way, that might come helpful to others, who are trying to achieve the same I wanted. Below is a sample working configuration:

var compassConfig = {
    options: {
        config: 'config.rb',
        bundleExec: true
    },
    compile: {
        options: {
            force: true
        }
    },
    watch: {
        options: {
            watch: true,
            force: true
        }
    }
};

var copyConfig = {
    src: {
        cwd: './',
        src: ['src/**/*', '!src/sass/**/*'],
        dest: 'dist/',
        expand: true
    }
}

var watchConfig = {
    options: {
        livereload: false,
        spawn: false
    },
    livereload: {
        files: ['dist/**/*'],
        options: {
            livereload: true,
            spawn: false
        }
    },
    src: {
        files: ['src/**/*', '!src/sass/**/*'],
        tasks: ['copy:src'],
    }
};

var connectConfig = {
    options: {
        base: 'dist/',
        middleware: function(connect, options, middlewares) {
            options.base = String(options.base);
            var express = require('express');
            var app = express();
            var directory = require('serve-index');
            app.use(express.static(options.base));
            app.get('/', function(req, res) {
                return res.render("" + options.base + "/index.html");
            });
            app.use(directory(options.base));
            middlewares.unshift(app);
            return middlewares;
        },
        port: 3333                        
      },
      dev: {
          options: {
              open: true,
              livereload: true,
          }
      }
    }
};

var concurrentConfig = {
    watch: {
        tasks: ['compass:watch', 'connect_watch', 'watch:src'],
        options: {
            logConcurrentOutput: true
        }
    }
};

grunt.initConfig({
    compass: compassConfig,
    concurrent: concurrentConfig,
    connect: connectConfig,
    copy: copyConfig,
    watch: watchConfig
});

grunt.registerTask('connect_watch', ['connect:dev', 'watch:livereload']);

grunt.registerTask('default', ['compass:compile', 'concurrent:watch']);

Note: There may be typo and the above code might not be directly copy-pasted as I've extracted the configuration from a much complex & large grunt configuration and simplified for brevity. Still, it should cover the usage pattern to solve this particular use case.