Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions doc/design/executor.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@

## Motivation

We use executor to do the runtime evaluation of a `ProgramDesc`.
We use the executor to do the runtime evaluation of a `ProgramDesc`.

## Overview

An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs.
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators in that block. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instances which are persistent throughout different runs.

### What does executor do?
### What does the executor do?

It evaluates all the operators in the `block_id`th block of a `ProgramDesc`.

### What does executor NOT do?
### What does the executor NOT do?

It does not do runtime optimization, meaning intelligently parse the dependency of each op a choose which one to be run and in which order they should be run.
It does not do runtime optimization which means it does not intelligently parse the dependency of each op. It also does not choose which op should be run and the order in which all the ops should be run.

It does not do graph partitioning, meaning dividing the `ProgramDesc` into several small pieces and executing them on different devices.
It does not do graph partitioning which means it does execute the process of dividing the `ProgramDesc` into several small pieces and executing them on different devices.

## Implementation

`Executor` evaluates a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then run all the operators in sequence. [[code]](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/executor.cc)
`Executor` evaluates a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence. [[code]](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/executor.cc)
20 changes: 10 additions & 10 deletions doc/design/functions_operators_layers.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Design Doc: Functions, Operators, and Layers

In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.
In a Deep Learning system, we can compose one or more fine grained operators into a coarse grained one. For example, the Fully Connected layer can be composed of a multiplication operator and an add operator.

Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.
Historically, some fine grained operations are known as operators, and the coarse ones are known as layers. But we need a well-defined separation.

In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
In general, operators are very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:

```c++
template <typename T> T add(T x, T y) { return x + y; }
template <typename T> T mul(T x, T y) { return x * y; }
```

Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation
Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation:

```c++
#define MAKE_FUNCTION_OPERATOR(mul);
Expand All @@ -24,7 +24,7 @@ template <typename T> class mulOp : public OperatorBase {...};
REGISTER_OP(mulOp<float32>, "mul");
```

so that in Python we can create operator mul by:
so that in Python we can create the operator mul by:

```python
X1 = Var()
Expand All @@ -33,7 +33,7 @@ Y = Var()
paddle.cpp.create_operator("mul", input=[X1, X2], output=Y)
```

Also, at the same time, we can compose a coarse level C++ operator class by composing functions `mul` and `add`:
At the same time, we can compose a coarse level C++ operator class by composing the functions `mul` and `add`:

```c++
template <typename T>
Expand All @@ -46,7 +46,7 @@ class FCOp : public OperatorBase {
REGISTER_OP(FCOp, "fc");
```

We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This higher level operator API should be compatible with the layer API.
We need to support such a composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This API should be compatible with the layer API.

Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`:

Expand All @@ -62,7 +62,7 @@ def operator.add(X1, X2):
return O
```

Above code snippets are automatically generated. Given them, users can define
the above code snippets are automatically generated. Given these, the users can define:

```python
def layer.fc(X):
Expand Down Expand Up @@ -96,5 +96,5 @@ We'd like to have Python bindings to operators in package `paddle.operator`, and

This is how we differentiate layer and operators in PaddlePaddle:

- those defined in C++ and have a lightweighted Python wrapper in module `operators` are operators; whereas
- those who don't have C++ implementations but a Python implementation that compose C++ operators are known as layers.
- those defined in C++ and have a light-weighted Python wrapper in module `operators` are operators.
- those who don't have C++ implementations but a Python implementation that composes C++ operators are known as layers.