Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[runtime] upgrade libtorch version to 2.1.0 #2418

Merged
merged 6 commits into from
Mar 21, 2024
Merged

[runtime] upgrade libtorch version to 2.1.0 #2418

merged 6 commits into from
Mar 21, 2024

Conversation

xingchensong
Copy link
Member

@xingchensong xingchensong commented Mar 18, 2024

try to fix #2417 && #2393

TODO

  • pass compilation
    • linux cpu
    • linux gpu
    • windows
    • macos
    • android libtorch (next PR)
  • wer verification
  • speed comparison with 1.13.0
libtorch aishell-1 test (chunk 16 left -1, ctc greedy search) rtf
2.1.0+cpu 5.87 % N=104765 C=98725 S=5892 D=148 I=107 0.04972 (int8)
2.1.0+gpu 5.80 % N=104765 C=98786 S=5833 D=146 I=101 0.07579 (fp32)
1.13.0+cpu 5.85 % N=104765 C=98746 S=5871 D=148 I=110 0.05211 (int8)

@xingchensong
Copy link
Member Author

why c++17?

img_v3_0293_4d9542d4-cab6-41ba-a21c-ea224d52d8ag

@Mddct
Copy link
Collaborator

Mddct commented Mar 19, 2024

https://github.com/wenet-e2e/wenet/blob/main/wenet/transformer/encoder.py#L265

记得这里 att mask to bias, sdpa的bool的mask 算的不太对(batch下不太对, 单条的不确定要不要to bias),

@xingchensong
Copy link
Member Author

https://github.com/wenet-e2e/wenet/blob/main/wenet/transformer/encoder.py#L265

记得这里 att mask to bias, sdpa的bool的mask 算的不太对(batch下不太对, 单条的不确定要不要to bias),

jit下用不到attention mask,哪个一直都是 torch.empty((0, 0, 0)),因为cache和current chunk都是valid

@xingchensong
Copy link
Member Author

Why have we chosen libtorch 2.1.0 over the 2.1.2 version, which is in strict alignment with the PyTorch version?

As of March 19, 2024, libtorch 2.1.0 represents the latest available release for download from the official repository at https://download.pytorch.org/libtorch/cpu

@Mddct Mddct merged commit 0e15804 into main Mar 21, 2024
6 checks passed
@Mddct Mddct deleted the xcsong-libtorch branch March 21, 2024 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

An error occurs when decoding with TLG model using JIT
2 participants