A Leiningen plugin for running tasks against a live Hadoop cluster. Adds a
profile named plugin.lein-hadoop-cluster/default
which includes the correct
class path etc for the currently-configured Hadoop cluster.
Put the following dependency into the :plugins
vector of your project.clj
:
[lein-hadoop-cluster "0.1.5"]
Due to technomancy/leiningen#1712, this currently will not work as a
profile-level plugin e.g. in your :user
profile. It must be a top-level
plugin in each project where you intend to use the plugin.
With the plugin added to the project, you can run tasks under the
plugin.lein-hadoop-cluster/default
profile, or otherwise use the profile to
access the live cluster task/JVM configuration. For example:
$ lein with-profile plugin.lein-hadoop-cluster/default trampoline repl
Internally, the plugin defines and makes use of two Leiningen profiles:
plugin.lein-hadoop-cluster/default
– Intended as a replacement for the top-leveldefault
. By inncludes the profilessystem
,base
,dev
,user
, andplugin.lein-hadoop-cluster/system
.plugin.lein-hadoop-cluster/system
– Additive profile containing the actual configuration necessary to access the system Hadoop installation.
Additionally, the plugin expects that project files will specify all their
Hadoop-provided dependencies via the provided
profile. Placing them in
e.g. dev
instead will result in them appearing on the cluster profile
classpaths before the system Hadoop, masking your live cluster configuration.
Copyright © 2014-2015 Marshall Bockrath-Vandegrift
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.