Various cleanups: remove start_ray_local from ray.init, remove unused code, fix "pip install numbuf". (ray-project#193)

* Remove start_ray_local from ray.init and change default number of workers to 10.

* Remove alexnet example.

* Move array methods to experimental.

* Remove TRPO example.

* Remove old files.

* Compile plasma when we build numbuf.

* Address comments.

Author: robertnishihara

README.md

@@ -17,8 +17,8 @@ machines).
 import ray
 import numpy as np
 
-# Start a scheduler, an object store, and some workers.
-ray.init(start_ray_local=True, num_workers=10)
+# Start Ray with some workers.
+ray.init(num_workers=10)
 
 # Define a remote function for estimating pi.
 @ray.remote
@@ -63,4 +63,3 @@ estimate of pi (waiting until the computation has finished if necessary).
 - [Hyperparameter Optimization](examples/hyperopt/README.md)
 - [Batch L-BFGS](examples/lbfgs/README.md)
 - [Learning to Play Pong](examples/rl_pong/README.md)
-- [Training AlexNet](examples/alexnet/README.md)

build-webui.sh

@@ -31,7 +31,7 @@ Python wrapper for an Atari simulator.
 import gym
 import ray
 
-ray.init(start_ray_local=True, num_workers=5)
+ray.init(num_workers=10)
 
 # Define a function to create the gym environment.
 def env_initializer():

doc/reusable-variables.md

@@ -76,7 +76,7 @@ This can be addressed by calling `ray.register_class(Foo)`.
 ```python
 import ray
 
-ray.init(start_ray_local=True, num_workers=1)
+ray.init(num_workers=10)
 
 # Define a custom class.
 class Foo(object):

doc/serialization.md

@@ -7,28 +7,37 @@ To use Ray, you need to understand the following:
 
 ## Overview
 
-Ray is a distributed extension of Python. When using Ray, several processes are
-involved.
-
-- A **scheduler**: The scheduler assigns tasks to workers. It is its own
-process.
-- Multiple **workers**: Workers execute tasks and store the results in object
-stores. Each worker is a separate process.
-- One **object store** per node: The object store enables the sharing of Python
-objects between worker processes so each worker does not have to have a separate
-copy.
-- A **driver**: The driver is the Python process that the user controls and
-which submits tasks to the scheduler. For example, if the user is running a
-script or using a Python shell, then the driver is the process that runs the
-script or the shell.
+Ray is a Python-based distributed execution engine. It can be used on a single
+machine to achieve effective multiprocessing, and it can be used on a cluster
+for large computations.
+
+When using Ray, several processes are involved.
+
+- Multiple **worker** processes execute tasks and store results in object stores.
+Each worker is a separate process.
+- One **object store** per node stores immutable objects in shared memory and
+allows workers to efficiently share objects on the same node with minimal
+copying and deserialization.
+- One **local scheduler** per node assigns tasks to workers on the same node.
+- A **global scheduler** receives tasks from local schedulers and assigns them
+to other local schedulers.
+- A **driver** is the Python process that the user controls. For example, if the
+user is running a script or using a Python shell, then the driver is the Python
+process that runs the script or the shell. A driver is similar to a worker in
+that it can submit tasks to its local scheduler and get objects from the object
+store, but it is different in that the local scheduler will not assign tasks to
+the driver to be executed.
+- A **Redis server** maintains much of the system's state. For example, it keeps
+track of which objects live on which machines and of the task specifications. It
+can also be queried directly for debugging purposes.
 
 ## Starting Ray
 
 To start Ray, start Python, and run the following commands.
 
 ```python
 import ray
-ray.init(start_ray_local=True, num_workers=10)
+ray.init(num_workers=10)
 ```
 
 That command starts a scheduler, one object store, and ten workers. Each of

doc/services-api.rst

@@ -79,7 +79,7 @@ import tensorflow as tf
 import numpy as np
 import ray
 
-ray.init(start_ray_local=True, num_workers=5)
+ray.init(num_workers=5)
 
 BATCH_SIZE = 100
 NUM_BATCHES = 1