-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Good First Issue]: [OP CONFORMANCE][TEMPLATE] One Multiply test in failed in confomance over template #22753
Comments
.take |
Thank you for looking into this issue! Please let us know if you have any questions or require any help. |
.take |
Thank you for looking into this issue! Please let us know if you have any questions or require any help. |
Hello @YutingGao7, are you still working on this? |
Hello @YutingGao7! What is the status of the issue? Please, let us know in case of any issues We will unassign the ticket from you in case of inaction |
.take |
Thank you for looking into this issue! Please let us know if you have any questions or require any help. |
@iefode I've done some research and I'm sharing the results with you: When executing a loop openvino/src/tests/functional/shared_test_classes/src/base/ov_subgraph.cpp Lines 75 to 78 in 6e9ee07
Iterations for The error occurs when allocating memory for the tensor I think it is necessary to change the logic of Shape generation to solve the problem. Line 51 in 6e9ee07
must according to the principle of "taking into account the available memory capacity when generating Shape". |
@iefode your feedback please |
@mlukasze Thanks for notification! @Vladislav-Denisov Many thanks for investigation! Looks like we should fix shape |
Context
OP conformance suite is validation tool checks a plugin conditions from operation implementation status perspective.
OP conformance is based on extracted from OMZ model scope operations and graphs by
ov_subgraphs_dumper
tool. Extracted graphs are saved as IR (OpenVINO Intermediate representation) and is stored in the public share.OP conformance suite contains the following test types:
OpImplCheck
validates operation support by plugin. Returntrue
orfalse
.Inference
compare device inference results vs reference over extracted IR as a model. We generate synthetic tensors to get a result. Possible results arepassed
,failed
,hanged
(interrupted by timeout),crashed
andskipped
(just in case-shape_mode
is misaligned with graph inputs).QueryModel
check possibility of graph execution per device. Have the same status asInference
ImportExport
exports compiled model, imports it and check that models are same. Have the same status asInference
Template is a simple plugin run inference using reference implementation. It means if we will run conformance over template, we should compare
TEMPLATE
vsTEMPLATE
results. Sometimes we have bugs inside the plugin, reference implementation or test infrastructure leads to negative test result.What needs to be done?
Description:
conformance_Multiply/ReadIRTest.Inference/Op=Multiply.1_Type=f32_Shape=dynamic_IR=287a7562757ef0295cc38442e3d775cff0fb1ea9b27e6897bd456f01ce82d455_Device=TEMPLATE_Config=()
is failed.How to reproduce:
ov_op_conformance_tests
targetExpected result:
Passed status for mentioned test.
Example Pull Requests
No response
Resources
Contact points
@iefode
Ticket
No response
The text was updated successfully, but these errors were encountered: