Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build sherpa-onnx as a single shared library #1078

Merged
merged 54 commits into from
Jul 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
a15f33c
Hide as many symbols as possible.
csukuangfj Jul 5, 2024
757b909
Fix for windows
csukuangfj Jul 5, 2024
66959bd
Fix windows building
csukuangfj Jul 5, 2024
3cc019c
Fix warnings on windows
csukuangfj Jul 5, 2024
8107d1b
restore BUILD_SHARED_LIBS
csukuangfj Jul 5, 2024
5329a80
minor fixes
csukuangfj Jul 5, 2024
66385cb
Fix ci
csukuangfj Jul 5, 2024
e08595d
Fix dart package
csukuangfj Jul 5, 2024
838116b
fix portaudio
csukuangfj Jul 5, 2024
2e24805
update espeak-ng
csukuangfj Jul 5, 2024
54b7c6a
remove static libs
csukuangfj Jul 5, 2024
f3e86fa
fix building node-addon
csukuangfj Jul 5, 2024
421517d
fix go tests
csukuangfj Jul 5, 2024
0297a20
fix node-addon
csukuangfj Jul 5, 2024
3399085
fix dart
csukuangfj Jul 5, 2024
d0b5be2
use a single job to build windows wheel
csukuangfj Jul 5, 2024
2800ebc
fix pkg-config
csukuangfj Jul 5, 2024
99d82f3
Fix rpath on Windows
csukuangfj Jul 5, 2024
c171f88
always build a single shared lib when BUILD_SHARED_LIBS in ON
csukuangfj Jul 5, 2024
045993d
do not install sherpa-onnx-core when BUILD_SHARED_LIBS is ON
csukuangfj Jul 5, 2024
a9d9098
remove rpath settings
csukuangfj Jul 5, 2024
d1a87d3
fix building on windows
csukuangfj Jul 5, 2024
6f88b39
Revert "remove rpath settings"
csukuangfj Jul 5, 2024
5eaab98
display wheel
csukuangfj Jul 5, 2024
94b95da
remove extra files from wheel
csukuangfj Jul 5, 2024
fdb18ba
fix building for dotnet and dart
csukuangfj Jul 5, 2024
ff18212
minor fixes
csukuangfj Jul 5, 2024
55d53c6
upload wheels
csukuangfj Jul 5, 2024
0d44016
minor fixes
csukuangfj Jul 5, 2024
264c090
fix releasing dart
csukuangfj Jul 6, 2024
416ad3e
Test go for Linux in CI
csukuangfj Jul 6, 2024
c002208
Test go for windows
csukuangfj Jul 6, 2024
2c1527e
fix go tests
csukuangfj Jul 6, 2024
58eb850
release v1.10.10
csukuangfj Jul 6, 2024
6da8e55
fix test go for windows
csukuangfj Jul 6, 2024
2da535d
fix dot net test
csukuangfj Jul 6, 2024
7fe757e
test dart for windows
csukuangfj Jul 6, 2024
bf41da3
minor fixes
csukuangfj Jul 6, 2024
4c11709
fix releasing windows
csukuangfj Jul 6, 2024
95cfa1b
fix releasing go
csukuangfj Jul 6, 2024
55bc14d
fix go
csukuangfj Jul 6, 2024
712d4f6
fix go
csukuangfj Jul 6, 2024
53e1a16
update go ci
csukuangfj Jul 6, 2024
d8f4c98
fix relesaing go
csukuangfj Jul 6, 2024
7e07ae4
disable testing dart for windows
csukuangfj Jul 6, 2024
4bca931
upload built libs
csukuangfj Jul 6, 2024
dc5c582
fix copy files
csukuangfj Jul 6, 2024
6d6a04b
fix install python
csukuangfj Jul 6, 2024
f97dbca
fix building for linux arm32 and arm64
csukuangfj Jul 6, 2024
27b3ec7
fix python tests
csukuangfj Jul 6, 2024
09a997f
fix installing portaudio
csukuangfj Jul 6, 2024
f5c0bf6
fix python tests
csukuangfj Jul 6, 2024
257a514
release v1.10.10
csukuangfj Jul 6, 2024
4bec77e
fix releasing go packages
csukuangfj Jul 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/scripts/test-dart.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,6 @@ cd dart-api-examples

pushd non-streaming-asr

echo '----------paraformer itn----------'
./run-paraformer-itn.sh

echo '----------paraformer----------'
./run-paraformer.sh
rm -rf sherpa-onnx-*

echo '----------VAD with paraformer----------'
./run-vad-with-paraformer.sh
rm -rf sherpa-onnx-*

echo '----------NeMo transducer----------'
./run-nemo-transducer.sh
rm -rf sherpa-onnx-*
Expand All @@ -37,6 +26,17 @@ echo '----------zipformer transducer----------'
./run-zipformer-transducer.sh
rm -rf sherpa-onnx-*

echo '----------paraformer itn----------'
./run-paraformer-itn.sh

echo '----------paraformer----------'
./run-paraformer.sh
rm -rf sherpa-onnx-*

echo '----------VAD with paraformer----------'
./run-vad-with-paraformer.sh
rm -rf sherpa-onnx-*

popd # non-streaming-asr

pushd tts
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/flutter-linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ concurrency:

jobs:
flutter_linux:
if: false
name: linux
runs-on: ${{ matrix.os }}
container: ubuntu:18.04
Expand All @@ -49,7 +50,7 @@ jobs:
shell: bash
run: |
apt-get update -y
apt-get install -y build-essential jq git cmake
apt-get install -y build-essential jq git cmake
apt-get install -y curl

- name: Setup Flutter SDK
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/jni.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, macos-14]
os: [ubuntu-latest, macos-latest, macos-13]

steps:
- uses: actions/checkout@v4
Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/linux-gpu.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,12 @@ jobs:

mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_INSTALL_PREFIX=./install -DBUILD_SHARED_LIBS=ON -DSHERPA_ONNX_ENABLE_GPU=ON ..
cmake \
-D CMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-D CMAKE_INSTALL_PREFIX=./install \
-D BUILD_SHARED_LIBS=ON \
-D SHERPA_ONNX_ENABLE_GPU=ON \
..

- name: Build sherpa-onnx for ubuntu
shell: bash
Expand All @@ -86,6 +91,12 @@ jobs:
ls -lh lib
ls -lh bin

echo "----"
ls -lh install/lib

echo "----"
ls -lh install/bin

- name: Display dependencies of sherpa-onnx for linux
shell: bash
run: |
Expand Down
14 changes: 13 additions & 1 deletion .github/workflows/linux-jni.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,13 @@ jobs:
mkdir build
cd build

cmake -DSHERPA_ONNX_ENABLE_TTS=ON -D CMAKE_BUILD_TYPE=Release -D BUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=./install -DSHERPA_ONNX_ENABLE_JNI=ON ..
cmake \
-D SHERPA_ONNX_ENABLE_TTS=ON \
-D CMAKE_BUILD_TYPE=Release \
-D BUILD_SHARED_LIBS=ON \
-D CMAKE_INSTALL_PREFIX=./install \
-D SHERPA_ONNX_ENABLE_JNI=ON \
..

make -j2
make install
Expand All @@ -105,6 +111,12 @@ jobs:
rm -rf ./install/pkgconfig
rm -rf ./install/share

echo "----"
ls -lh install/lib

echo "----"
ls -lh install/bin

- name: Display dependencies of sherpa-onnx for linux
shell: bash
run: |
Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,14 +99,25 @@ jobs:
mkdir build
cd build

cmake -DSHERPA_ONNX_ENABLE_TTS=${{ matrix.with_tts }} -D CMAKE_BUILD_TYPE=${{ matrix.build_type }} -D BUILD_SHARED_LIBS=${{ matrix.shared_lib }} -DCMAKE_INSTALL_PREFIX=./install ..
cmake \
-D SHERPA_ONNX_ENABLE_TTS=${{ matrix.with_tts }} \
-D CMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-D BUILD_SHARED_LIBS=${{ matrix.shared_lib }} \
-D CMAKE_INSTALL_PREFIX=./install \
..

make -j2
make install

ls -lh lib
ls -lh bin

echo "----"
ls -lh install/lib

echo "----"
ls -lh install/bin

- name: Display dependencies of sherpa-onnx for linux
shell: bash
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/macos-jni.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ jobs:
cmake \
-D BUILD_SHARED_LIBS=ON \
-D CMAKE_BUILD_TYPE=Release \
-DCMAKE_OSX_ARCHITECTURES=$arch \
-DSHERPA_ONNX_ENABLE_JNI=ON \
-D CMAKE_OSX_ARCHITECTURES=$arch \
-D SHERPA_ONNX_ENABLE_JNI=ON \
-DCMAKE_INSTALL_PREFIX=./install \
..

Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,13 @@ jobs:
BUILD_SHARED_LIBS=ON
fi

cmake -DSHERPA_ONNX_ENABLE_TTS=${{ matrix.with_tts }} -D BUILD_SHARED_LIBS=$BUILD_SHARED_LIBS -D CMAKE_BUILD_TYPE=${{ matrix.build_type }} -DCMAKE_OSX_ARCHITECTURES='arm64;x86_64' -DCMAKE_INSTALL_PREFIX=./install ..
cmake \
-DSHERPA_ONNX_ENABLE_TTS=${{ matrix.with_tts }} \
-D BUILD_SHARED_LIBS=$BUILD_SHARED_LIBS \
-D CMAKE_BUILD_TYPE=${{ matrix.build_type }} \
-D CMAKE_OSX_ARCHITECTURES='arm64;x86_64' \
-D CMAKE_INSTALL_PREFIX=./install \
..

- name: Build sherpa-onnx for macos
shell: bash
Expand Down
23 changes: 5 additions & 18 deletions .github/workflows/release-dart-package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -93,17 +93,12 @@ jobs:
make install

ls -lh ./install/lib
rm -v ./install/lib/libonnxruntime.so

- name: Copy pre-built libs
shell: bash
run: |
cp -v build/install/lib/lib*.so* flutter/sherpa_onnx_linux/linux/
pushd flutter/sherpa_onnx_linux/linux/

rm libonnxruntime.so
ln -s libonnxruntime.so.* ./libonnxruntime.so

popd

mv -v flutter/sherpa_onnx_linux /tmp/to_be_published

Expand Down Expand Up @@ -197,27 +192,19 @@ jobs:
cd build
make -j2 install

ls -lh install/lib/libsherpa-onnx-core.dylib
file install/lib/libsherpa-onnx-core.dylib
ls -lh install/lib/libsherpa-onnx-c-api.dylib
file install/lib/libsherpa-onnx-c-api.dylib
rm -v install/lib/libonnxruntime.dylib

- name: Copy pre-built libs
shell: bash
run: |
cp -v build/install/lib/lib*.dylib* flutter/sherpa_onnx_macos/macos/


mv -v flutter/sherpa_onnx_macos /tmp/to_be_published

ls -lh /tmp/to_be_published/macos

pushd /tmp/to_be_published/macos

rm libonnxruntime.dylib

ls -lh

popd

- name: Release
shell: bash
run: |
Expand Down Expand Up @@ -283,7 +270,7 @@ jobs:
shell: bash
run: |
cd build
cmake --build . --target install --config Release
cmake --build . --target install --config Release -- -m:2

ls -lh install/lib/*.dll

Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/riscv64-linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,16 @@ jobs:
ls -lh build-riscv64-linux-gnu/bin
ls -lh build-riscv64-linux-gnu/lib

echo "---install/lib---"
ls -lh build-riscv64-linux-gnu/install/lib

echo "---install/bin---"
ls -lh build-riscv64-linux-gnu/install/bin

file build-riscv64-linux-gnu/bin/sherpa-onnx

readelf -d build-riscv64-linux-gnu/bin/sherpa-onnx

- name: Test sherpa-onnx
shell: bash
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/run-python-test-macos.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
- name: Install Python dependencies
shell: bash
run: |
python3 -m pip install --upgrade pip numpy pypinyin sentencepiece>=0.1.96 soundfile
python3 -m pip install --upgrade pip numpy pypinyin sentencepiece>=0.1.96 soundfile setuptools wheel

- name: Install sherpa-onnx
shell: bash
Expand All @@ -87,7 +87,7 @@ jobs:
export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
cmake --version

python3 setup.py install
python3 -m pip install .

- name: Test sherpa-onnx
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/swift.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [macos-latest, macos-14]
os: [macos-latest, macos-13]

steps:
- uses: actions/checkout@v4
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/test-build-wheel.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,20 @@ jobs:
python3 setup.py bdist_wheel
ls -lh dist

- uses: actions/upload-artifact@v4
with:
name: wheel-${{ matrix.os }}-${{ matrix.python-version }}
path: ./dist/*.whl

- name: Display wheel
shell: bash
run: |
ls -lh dist
cd dist

mkdir t
cd t
unzip ../*.whl

- name: Install wheel
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-dart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ jobs:
cd build

cmake \
-D BUILD_SHARED_LIBS=ON \
-DBUILD_SHARED_LIBS=ON \
-DSHERPA_ONNX_ENABLE_PORTAUDIO=OFF \
-DSHERPA_ONNX_ENABLE_WEBSOCKET=OFF \
-DBUILD_ESPEAK_NG_EXE=OFF \
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/test-go-package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@ jobs:
include:
- os: ubuntu-latest
arch: amd64
- os: macos-latest
- os: macos-13
arch: amd64
- os: macos-14
arch: arm64
- os: windows-latest
arch: x64
- os: windows-latest
Expand Down
Loading
Loading