Skip to content

Commit 77a29b5

Browse files
GregoryComerGregoryComer
committed
WIP New getting started
1 parent a5c7609 commit 77a29b5

File tree

2 files changed

+195
-1
lines changed

2 files changed

+195
-1
lines changed

‎docs/source/getting-started.md‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# Getting Started with ExecuTorch
2+
This section is intended to describe the necessary steps to take PyTorch model anda run it using ExecuTorch. To use the framework, you will need to take the following steps:
3+
- Install the ExecuTorch python package and runtime libraries.
4+
- Export the PyTorch model for the target hardware configuration.
5+
- Run the model using the ExecuTorch runtime APIs.
6+
7+
<hr/>
8+
9+
## Installation
10+
To use ExecuTorch, you will need to install both the Python package and the appropriate platform-specific runtime libraries.
11+
12+
Pip is the recommended way to install the ExecuTorch python package. This package includes the dependencies needed to export a PyTorch model, as well as Python runtime bindings for model testing and evaluation.
13+
```
14+
pip install executorch
15+
```
16+
17+
#### System Requirements
18+
- Python 3.10 - 3.12
19+
- g++ version 7 or higher, clang++ version 5 or higher, or another C++17-compatible toolchain.
20+
- Linux or MacOS operating system (ARM or x86).
21+
- Windows is supported via WSL.
22+
23+
#### Runtime Libraries
24+
ExecuTorch provides a native C++ runtime API, as well as platform and language-specific bindings for Python, Android, and iOS.
25+
26+
##### Python
27+
Python runtime bindings are provided as part of the executorch pip package.
28+
29+
##### Android
30+
ExecuTorch Android Java libraries are available as a pre-built library. To add the library to your app, download the AAR, and add it to the gradle build rule. TODO Replace with Maven/Gradle package management when available.
31+
```
32+
mkdir -p app/libs
33+
curl https://ossci-android.s3.amazonaws.com/executorch/release/executorch-241002/executorch.aar -o app/libs/executorch.aar
34+
```
35+
And in gradle,
36+
```
37+
# app/build.gradle.kts
38+
dependencies {
39+
implementation(files("libs/executorch.aar"))
40+
}
41+
```
42+
43+
##### iOS
44+
The iOS runtime library is provided as a collection of .xcframework targets and are made available as a Swift PM package. To get started with Xcode, go to File > Add Package Dependencies. Paste the URL of the ExecuTorch repo into the search bar and select it. Make sure to change the branch name to the desired ExecuTorch version in format “swiftpm-”, (e.g. “swiftpm-0.4.0”), or a branch name in format “swiftpm-.<year_month_date>” (e.g. “swiftpm-0.4.0-20241201”) for a nightly build on a specific date.
45+
46+
The ExecuTorch dependency can also be added to the package file manually. See [Using ExecuTorch on iOS](/TODO.md) for more information.
47+
48+
##### C++
49+
50+
CMake is the preferred build system for the ExecuTorch C++ runtime. To use with CMake, clone the ExecuTorch repository as a subdirectory of your project, and use CMake's `add_subdirectory("executorch")` to include the dependency. The `executorch` target, as well as kernel and backend targets will be made available to link against. The runtime can also be built standalone to support diverse toolchains. See [Using ExecuTorch with C++](/TODO.md) for a detailed description of build integration, targets, and cross compilation.
51+
52+
```
53+
git clone -b release/0.5 https://github.com/pytorch/executorch.git
54+
```
55+
```python
56+
# CMakeLists.txt
57+
add_subdirectory("executorch")
58+
...
59+
target_link_libraries(
60+
my_target
61+
PRIVATE executorch
62+
executorch_module_static
63+
executorch_tensor
64+
optimized_native_cpu_ops_lib
65+
xnnpack_backend)
66+
```
67+
68+
<hr/>
69+
70+
## Preparing the Model
71+
Exporting is the process of taking a PyTorch model and converting it to the .pte file format used by the ExecuTorch runtime. This is done using Python APIs. PTE files for common models can be found on HuggingFace (TODO add link).
72+
73+
### Requirements
74+
- A PyTorch model.
75+
- Example model inputs, typically as PyTorch tensors. You should be able to successfully run the PyTorch model with these inputs.
76+
- One or more target hardware backends.
77+
78+
### Selecting a Backend
79+
ExecuTorch provides hardware backends for a wide variety of hardware. The most commonly used backends are XNNPACK, for ARM and x86 CPU, CoreML (for iOS), and Vulkan (for Android GPUs).
80+
81+
For mobile, consider using XNNPACK for Android and CoreML or XNNPACK for iOS.
82+
83+
See [Delegates](/TODO.md) for a description of available backends.
84+
85+
### Exporting
86+
Exporting is done using Python APIs. ExecuTorch provides a high degree of customization during the export process, but the typical flow is as follows:
87+
```python
88+
model = MyModel() # The PyTorch model to export
89+
example_inputs = (torch.randn(1,3,64,64),) # A tuple of inputs
90+
91+
et_program =
92+
executorch.exir.to_edge_transform_and_lower(
93+
torch.export.export(model, example_inputs)
94+
partitioner=[XnnpackPartitioner()]
95+
).to_executorch()
96+
97+
with open(“model.pte”, “wb”) as f:
98+
f.write(et_program.buffer)
99+
```
100+
101+
If the model requires varying input sizes, you will need to specify the varying dimensions and bounds as part of the `export` call. See [Exporting a Model for ExecuTorch](/TODO.md) for more information.
102+
103+
The hardware backend to target is controlled by the partitioner parameter to to\_edge\_transform\_and\_lower. In this example, the XnnpackPartitioner is used to target mobile CPUs. See the delegate-specific documentation for a full description of the partitioner and available options.
104+
105+
Quantization can also be done at this stage to reduce model size and runtime. Quantization is backend-specific. See the documentation for the target backend for a full description of supported quantization schemes.
106+
107+
After successfully generating a .pte file, it is common to use the Python runtime bindings to validate the model. This can be used to evaluate model accuracy before running on-device.
108+
109+
<hr/>
110+
111+
## Running the Model
112+
ExecuTorch provides Python, Java, Objective-C, and C++ APIs. The Python APIs are intended for testing and model evaluation and are not intended to be used for production deployments.
113+
114+
### Python
115+
The ExecuTorch Python bindings are included with the ExecuTorch pip package. See Installing the ExecuTorch Python Libraries (TODO link) for information on how to install the package.
116+
117+
Inference can be run as follows:
118+
```python
119+
from executorch.extensions.pybindings.portable_lib import _load_for_executorch
120+
121+
input_tensor = torch.randn(1,3,128,128)
122+
model = _load_for_executorch(“/path/to/model.pte”)
123+
outputs = model(input_tensor)
124+
```
125+
126+
### Android
127+
ExecuTorch provides Java bindings for Android usage, which can be consumed from both Java and Kotlin. See the [Installation](/TODO.md) section above for instructions on adding the dependency. Models can be loaded and run as follows:
128+
```java
129+
import org.pytorch.executorch.EValue
130+
import org.pytorch.executorch.Module
131+
import org.pytorch.executorch.Tensor
132+
133+
// …
134+
135+
Module model = Module.load(“/path/to/model.pte”)
136+
// TODO Add input setup
137+
EValue output = model.forward(input_evalue);
138+
```
139+
140+
For more information on Android development, including building from source, a full description of the Java APIs, and information on using ExecuTorch from Android native code, see [Using ExecuTorch on Android](/TODO.md).
141+
142+
### iOS
143+
ExecuTorch supports both iOS and MacOS via C++ and Objective-C bindings, as well as hardware backends for CoreML, MPS, and CPU. Binaries for the runtime are provided as a Swift PM package. See the [Installation](/TODO.md) section above for instructions on adding the dependency.
144+
145+
Models can be loaded and run from Swift as follows:
146+
```swift
147+
// TODO Code sample
148+
```
149+
150+
For more information on iOS integration, including an API reference, logging setup, and building from source, see [Using ExecuTorch on iOS](/TODO.md).
151+
152+
### C++
153+
ExecuTorch provides C++ APIs, which can be used to target embedded or mobile devices. The C++ APIs provide a greater level of control compared to other language bindings, allowing for advanced memory management, data loading, and platform integration.
154+
155+
Both high-level and low-level C++ APIs are provided. The low-level APIs are platform independent, do not dynamically allocate memory, and are most suitable for resource-constrained embedded systems. The high-level APIs are provided as a convenience wrapper around the lower-level APIs, and make use of dynamic memory allocation and standard library constructs to reduce verbosity.
156+
157+
ExecuTorch uses CMake for native builds. Integration is typically done by cloning the ExecuTorch repository and using CMake add_subdirectory to add the dependency.
158+
159+
Loading and running a model using the high-level API can be done as follows:
160+
```cpp
161+
#include <executorch/extension/module/module.h>
162+
#include <executorch/extension/tensor/tensor.h>
163+
164+
using namespace ::executorch::extension;
165+
166+
// Load the model.
167+
Module module("/path/to/model.pte");
168+
169+
// Create an input tensor.
170+
float input[1 * 3 * 256 * 256];
171+
auto tensor = from_blob(input, {1, 3, 256, 256});
172+
173+
// Perform an inference.
174+
const auto result = module.forward(tensor);
175+
176+
if (result.ok()) {
177+
// Retrieve the output data.
178+
const auto output = result->at(0).toTensor().const_data_ptr<float>();
179+
}
180+
```
181+
182+
For more information on the C++ APIs, see [Using ExecuTorch with C++](/TODO.md).
183+
184+
<hr/>
185+
186+
## Next Steps
187+
ExecuTorch provides a high-degree of customizability to support diverse hardware targets. Depending on your use cases, consider exploring one or more of the following pages:
188+
189+
- [Exporting a Model to ExecuTorch](/TODO.md) for advanced model conversion options.
190+
- [Delegates](/TODO.md) for available backends and configuration options.
191+
- [Using ExecuTorch on Android](/TODO.md) and [Using ExecuTorch on iOS](TODO.md) for mobile runtime integration.
192+
- [Using ExecuTorch with C++](/TODO.md) for embedded and mobile native development.
193+
- [Troubleshooting, Profiling, and Optimization](/TODO.md) for developer tooling and debugging.
194+
- [API Reference](/TODO.md) for a full description of available APIs.

‎docs/source/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ Topics in this section will help you get started with ExecuTorch.
8484
:caption: Getting Started
8585
:hidden:
8686

87-
getting-started-setup
87+
getting-started
8888
export-overview
8989
runtime-build-and-cross-compilation
9090
getting-started-faqs

0 commit comments

Comments
 (0)