Travis CI integration

[mark-rushakoff]

This commit includes everything necessary to run the test suite in PhantomJs on Travis CI.

You can see a green build at http://travis-ci.org/#!/mark-rushakoff/backbone

[braddunbar]

I'm not sure where @jashkenas stands on this, but if we're going to run tests from the console I'd rather it be rolled up in something like grunt so that we aren't maintaining it. Will grunt run on travis? If so, can we just include it in package.json and point travis to a script that does grunt qunit?

[mark-rushakoff]

I don't know enough about node/npm to say so definitively, but it does look like you would be able to just drop grunt in (via package.json) and run qunit.

I didn't know of grunt until just now, and glancing over its code, I feel like he's offering a more full-featured suite of tools at the cost of more configuration. On the other hand, your concern about maintenance is very valid, and I haven't yet solved that problem in my project (OpenPhantomScripts, from which the code in this pull request came).

I won't be offended if you choose grunt over my project -- I'd just really like to see that Travis "build: passing" badge on the front page of Underscore and Backbone.

In the meantime, I'm going to do some research into how to alleviate the maintenance concern you have with my code (if anyone has any comments, please chime in on the issue on OpenPhantomScripts).

[jashkenas]

I'm a little confused -- why would we want to test Backbone outside of a browser? ... given that it's intended to run in the browser, and we already test commits and patches in browsers. What do we gain?

[tbranyen]

@jashkenas headless testing with grunt would be nice since its something you get "for free", it takes no added work other than pointing to the qunit folder and then you could put that in front of the min process to stop it from building if there are any errors.

i don't see it as anything other than convenience.

[mark-rushakoff]

PhantomJs is a headless webkit browser. It should behave (roughly) the same as Safari or Chrome. So running the tests in Phantom isn't "outside a browser", it's "in another browser."

While I do believe that you do manually test commits, by not having a published CI, I feel like you're asking the consumers of Underscore and Backbone to take it on faith that the last commit has been tested. Or you're asking us to run the tests ourselves before we put the library in our code.

When I see the green "build passed" badge on a Github project, I am reasonably assured that the code from the latest commit works as intended.

Maybe a compromise is to put a direct link to the tests in the README?

[jashkenas]

Yep -- there are direct links to the live test pages at the top of the homepage (which serves as the real README).

When I see the green "build passed" badge on a Github project, I am
reasonably assured that the code from the latest commit works as intended.

... I think you're putting overmuch faith in the existence of tests then. Our releases always pass the test suite in major browsers, but we nonetheless frequently have bugs in untested edge cases.

[jashkenas]

Reopening ...

Now that GitHub pull requests support Travis badges, there's a pretty solid benefit to potentially adding Travis support.

https://github.com/blog/1227-commit-status-api

[philfreo]

+1 for using grunt qunit. grunt could also replace the Rakefile so the project is fully JS-based.

http://gruntjs.com/