Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ort benchmark script in xp folder #139

Merged
merged 4 commits into from
Nov 2, 2022

Conversation

pommedeterresautee
Copy link
Member

fix #115

@pommedeterresautee pommedeterresautee added the benchmark Measure, measure, measure label Oct 31, 2022
@pommedeterresautee pommedeterresautee self-assigned this Oct 31, 2022
@pommedeterresautee pommedeterresautee marked this pull request as ready for review October 31, 2022 21:01
@pommedeterresautee
Copy link
Member Author

test pass

=========================================================================================================== warnings summary ===========================================================================================================
conftest.py:41
  /mnt/workspace/kernl/conftest.py:41: PytestDeprecationWarning: The hookimpl pytest_configure uses old-style configuration options (marks or attributes).
  Please use the pytest.hookimpl(trylast=True) decorator instead
   to configure the hooks.
   See https://docs.pytest.org/en/latest/deprecations.html#configuring-hook-specs-impls-using-markers
    @pytest.mark.trylast

test/test_debugger.py::test_matmul
  /mnt/workspace/kernl/test/test_debugger.py:172: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
    group_id = pid // num_pid_in_group

test/test_debugger.py::test_matmul
  /mnt/workspace/kernl/test/test_debugger.py:176: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
    pid_n = (pid % num_pid_in_group) // group_size_m

test/test_torchdynamo.py::test_t5
  /home/geantvert/.local/share/virtualenvs/kernl/lib/python3.9/site-packages/transformers/models/t5/tokenization_t5_fast.py:156: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
  For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
  - Be aware that you SHOULD NOT rely on t5-small automatically truncating your input to 512 when padding/encoding.
  - If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
  - To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================================== 2461 passed, 356 skipped, 4 warnings in 7015.64s (1:56:55) ======================================================================================

@pommedeterresautee pommedeterresautee merged commit 9c0159e into main Nov 2, 2022
@pommedeterresautee pommedeterresautee deleted the feat/onnxruntime_benchmark branch November 2, 2022 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Measure, measure, measure feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move Onnx test inside experimental folder
2 participants