Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cpp/README.md #2980

Merged
merged 3 commits into from
Feb 29, 2024
Merged

Update cpp/README.md #2980

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1dddcee
Update cpp/README.md
mreso Feb 28, 2024
b38126d
Fix spell check
mreso Feb 28, 2024
aa81c26
Merge branch 'master' into docs/cpp_readme
agunapal Feb 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions cpp/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ cd cpp
cd ..
python ts_scripts/install_from_src.py
```
### Set Environment Var
#### On Mac
```
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$(pwd)/_build/_deps/libtorch/lib
```
#### On Ubuntu
```
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/_build/_deps/libtorch/lib
```

### Run TorchServe
```
mkdir model_store
Expand Down Expand Up @@ -67,3 +77,23 @@ This will make the TorchServe installation editable and the updated cpp backend
```
python ts_scripts/install_from_src.py --environment dev
```

#### FAQs
Q: After running ./build.sh TorchServe can not find model_worker_socket
A:
1. See if the binary `model_worker_socket` exists by running:
```bash
python -c "import ts; from pathlib import Path; print((Path(ts.__file__).parent / 'cpp/bin/model_worker_socket').exists())
```
2. Look if ./build.sh was actually successful and if the tests ran without any error at the end. If a test failed the binary will not be copied into the appropriate directory.
3. Make sure you have the right conda/venv environment activated during building that you're also using to run TorchServe.

Q: Build on Mac fails with `Library not loaded: @rpath/libomp.dylib`
A: Install libomp with brew and link in /usr/local/lib
```bash
brew install libomp
sudo ln -s /opt/homebrew/opt/libomp/lib/libomp.dylib /usr/local/lib/libomp.dylib
```

Q: When loading a handler which uses a model exported with torch._export.aot_compile the handler dies with "error: Error in dlopen: MODEL.SO : undefined symbol: SOME_SYMBOL".
A: Make sure that you are using matching libtorch and Pytorch versions for inference and export, respectively.
Loading