Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEST] Compare whole Tensor in Template Plugin tests #23408

Conversation

t-jankowski
Copy link
Contributor

@t-jankowski t-jankowski commented Mar 12, 2024

Details:

  • ov::test::utils::compare(...) used for output vs reference values comparison (where possible).
  • Tests with a need of complex changes were marked with legacy_compare flag. These tests still use legacy raw data comparison and theirs transition will be addressed in further PR's.

Tickets:

@github-actions github-actions bot added the category: TEMPLATE OpenVINO Template plugin label Mar 12, 2024
@t-jankowski
Copy link
Contributor Author

Blocked by #23226

@t-jankowski t-jankowski requested review from olpipi and vurusovs March 13, 2024 08:42
@mlukasze mlukasze added this to the 2024.1 milestone Mar 20, 2024
@t-jankowski t-jankowski marked this pull request as ready for review March 25, 2024 11:55
@t-jankowski t-jankowski requested a review from a team as a code owner March 25, 2024 11:55
@mlukasze mlukasze enabled auto-merge March 27, 2024 05:56
"tile_1d_to_3d_with_zero_on_axis_0"),
TileParams(reference_tests::Tensor(ET, {3}, std::vector<T>{1, 2, 3}),
reference_tests::Tensor(ET_INT, {3}, std::vector<T_INT>{2, 0, 3}),
reference_tests::Tensor(ET, {0}, std::vector<T>{}),
reference_tests::Tensor(ET, {2, 0, 9}, std::vector<T>{}),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why it is required? I expected tensor with no values should have shape == {0}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well .. shapes comparison fails otherwise. Should the comparison be improved? If so, I'll move this test back to legacy raw compare.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However this shape is produced by shape_infer, so it should be consider valid. @praasz, what do think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tensor can have shape set with dimension 0 which will result as no data but still can have some dimension.

If these test validate all properties of the tensor like type, shape, data then it should be set what shape inference calculate for this operator (in this case Tile should not reduce rank.
There will be just no data to calculate/compare.

In my opinion this is OK as we check what Tile produce.

Maybe is not real scenario but Is possible to make this kind of Tile as sub-graph and gather first and/or last dimension(s) from Tile output make ShapeOf of it which can produce data like {2,9} which can be input for other operator.

@mlukasze mlukasze requested a review from vurusovs March 29, 2024 05:48
@iefode iefode self-assigned this Apr 2, 2024
@mlukasze mlukasze added this pull request to the merge queue Apr 3, 2024
Merged via the queue into openvinotoolkit:master with commit 0799163 Apr 3, 2024
108 checks passed
bbielawx pushed a commit to bbielawx/openvino that referenced this pull request Apr 12, 2024
…#23408)

### Details:
- `ov::test::utils::compare(...)` used for output vs reference values
comparison (where possible).
- Tests with a need of complex changes were marked with `legacy_compare`
flag. These tests still use legacy raw data comparison and theirs
transition will be addressed in further PR's.

### Tickets:
 - CVS-135274, CVS-135275

---------

Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
alvoron pushed a commit to alvoron/openvino that referenced this pull request Apr 29, 2024
…#23408)

### Details:
- `ov::test::utils::compare(...)` used for output vs reference values
comparison (where possible).
- Tests with a need of complex changes were marked with `legacy_compare`
flag. These tests still use legacy raw data comparison and theirs
transition will be addressed in further PR's.

### Tickets:
 - CVS-135274, CVS-135275

---------

Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: TEMPLATE OpenVINO Template plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants