Skip to content

liyinhgqw/piccolo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

-- OVERVIEW

-- PREREQUISITES

To build and use Piccolo, you will need a minimum of the following:

* CMake (> 2.6)
* OpenMPI
* Python (2.*)
* gcc/g++ (> 4)
* protocol buffers

If available, the following libraries will be used:

* Python development headers; SWIG
* TCMalloc
* google-perftools

In addition to these, Piccolo comes with several support libraries which are 
compiled as part of the build process; these are:

* google-flags
* google-logging


On debian/ubuntu, the required libraries can be acquired by running:

sudo apt-get install\
 build-essential\
 cmake\
 g++\
 libboost-dev\
 libboost-python-dev\
 libboost-thread-dev\
 liblzo2-dev\
 libnuma-dev\
 libopenmpi-dev\
 libprotobuf-dev \
 libcr-dev\
 libibverbs-dev\
 openmpi-bin\
 protobuf-compiler\
 
the optional libraries can be install via:

sudo apt-get install libgoogle-perftools-dev python-dev swig

-- BUILDING

To build, simply run 'make' from the toplevel piccolo directory.  After building
output should be available in the build/ directory.  Specifically, a successful
build should generate an executable for each example case:

build/
      accelpagerank
      bipartmatch
      bipartmatch-trigger
      faceclass
      k-means
      matmul
      pagerank
      shortest-path
      shortest-path-trigger
      test-tables
      test-tables2
      wordcount


-- RUNNING

To execute a Piccolo program, you will need to modify conf/mpi-cluster
to point to the set of machines Piccolo will be executed on - for example, a file
might look like:

localhost slots=1
a slots=4
b slots=4
c slots=4

Which allows for running up to 12 workers (+ 1 master process).

In addition to the MPI configuration, LD_LIBRARY_PATH must be set to 
allow the Piccolo .so files to be located; this can be done via:

You can run an example via:

LD_LIBRARY_PATH=build/ \
 build/k-means \
 --num_points=10000 \
 --num_clusters=100 \
 --workers=12 \
 --hostfile=conf/mpi-cluster 
 
This will create and test a simple k-means clustering experiment using
the workers specified.

-- DEVELOPMENT

For more information on how to create your own programs, check out the
API documentation.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published