Ignition-Template

This provides a template to work with Apache Spark and with Ignition-Core.

Notable features

• This template will point to a stable Spark and Hadoop-client version, potentially working-around some known bugs
• src/main/scala/ignition/jobs/Runner.scala is a example of multi-setup project that can run using core/tools/cluster.py jobs run command
• See scripts/job_runner.py for a full-featured, production-grade script-template that launchs and supervises clusters and jobs using AWS spot instances with automatic failure recovery

In general, check the source files for more information.

Getting started

See http://monkeys.chaordic.com.br/start-using-spark-with-ignition/ for quick-start tutorial

TODO

• An example using Scalaz Validation
• Other examples, more documentation