Skip to content

Implement ScatterND operation in MO and transform SparseToDense for W&D model #584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

rkazants
Copy link
Member

@rkazants rkazants commented May 26, 2020

Description:

  • SparseToDense used in Wide and Deep model is expressed through ScatterND operation. ScatterND is more functional than SparseToDense. Hence, it was decided to replace SparseToDense with ScatterND. ScatterND is more useful for other models.
  • Removed SparseToDense from the previous opset.
  • Updated opset for ScatterNDUpdate, now it is opset4.
  • Updated IR Reader to be able to read ScatterNDBase, EmbeddingBag inherrited operations

JIRA: CVS-30357

Code:

  • Comments
  • Code style (PEP8)
  • Transformation generates reshape-able IR
  • Transformation preserves original framework node names

Validation:

  • Unit tests
  • Framework operation tests - layer test for SparseToDense will be done in the scope of CVS-34789
  • Transformation tests
  • e2e model test with an update of ./tests/e2e_oss/utils/reshape_utils.py
  • Model Optimizer IR Reader check - checked for W&D with new op - ScatterNDUpdate

Documentation:

  • Supported frameworks operations list - updated with SparseToDense
  • Supported public models list - updated with W&D
  • New operations specification - done for ScatternNDUpdate, SparseToDense was experimental operation so it doesn't exist in the spec
  • Guide on how to convert the public model - The guide to convert W&D model exists
  • User guide update - No update needed. Usage of W&D model is not changed

Signed-off-by: Roman Kazantsev roman.kazantsev@intel.com

@rkazants rkazants requested a review from a team May 26, 2020 08:57
@rkazants rkazants added the category: MO Model Optimizer label May 26, 2020
@e-nugmanova
Copy link
Contributor

Please fill in the description. If PR is not ready for review -- please convert it to draft.

@rkazants rkazants force-pushed the feature/rkazants/scatternd branch from 2eda5bd to b5a9fd4 Compare July 9, 2020 07:00
@rkazants rkazants requested a review from a team July 9, 2020 07:00
@rkazants rkazants changed the title [DO NOT MERGE] Implement ScatterND operation in MO and transform for SparseToDense Implement ScatterND operation in MO and transform SparseToDense for W&D model Jul 9, 2020
@rkazants rkazants force-pushed the feature/rkazants/scatternd branch from b5a9fd4 to e87f63b Compare July 9, 2020 08:31
@rkazants rkazants requested a review from a team July 9, 2020 08:31
@rkazants rkazants force-pushed the feature/rkazants/scatternd branch from e87f63b to 8760033 Compare July 9, 2020 09:11
@rkazants
Copy link
Member Author

rkazants commented Jul 9, 2020

@lazarevevgeny, kindly reminder to review ScatterNDUpdate in MO implementation.

Copy link
Contributor

@lazarevevgeny lazarevevgeny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The review quality is very good! Most of the comments are minor.

@rkazants
Copy link
Member Author

The review quality is very good! Most of the comments are minor.

Yes, quality of your review is definitely very good as always :)

SparseToDense used in Wide and Deep model is expressed through ScatterND operation.
ScatterND is more functional than SparseToDense. Hence, it was decided to replace SparseToDense
with ScatterND. ScatterND is more useful for other models.

Remove SparseToDense from the previous opset

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
@rkazants rkazants force-pushed the feature/rkazants/scatternd branch from 8760033 to ca60eaf Compare July 10, 2020 07:31
@lazarevevgeny lazarevevgeny merged commit 7b65ba3 into openvinotoolkit:master Jul 10, 2020
redradist pushed a commit to redradist/openvino that referenced this pull request Oct 6, 2023
github-merge-queue bot pushed a commit that referenced this pull request May 9, 2025
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 2.11.1
to 6.1.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst">pytest-cov's
changelog</a>.</em></p>
<blockquote>
<h2>6.1.1 (2025-04-05)</h2>
<ul>
<li>Fixed breakage that occurs when <code>--cov-context</code> and the
<code>no_cover</code> marker are used together.</li>
</ul>
<h2>6.1.0 (2025-04-01)</h2>
<ul>
<li>Change terminal output to use full width lines for the coverage
header.
Contributed by Tsvika Shapira in
<code>[#678](pytest-dev/pytest-cov#678)
&lt;https://github.com/pytest-dev/pytest-cov/pull/678&gt;</code>_.</li>
<li>Removed unnecessary CovFailUnderWarning. Fixes
<code>[#675](pytest-dev/pytest-cov#675)
&lt;https://github.com/pytest-dev/pytest-cov/issues/675&gt;</code>_.</li>
<li>Fixed the term report not using the precision specified via
<code>--cov-precision</code>.</li>
</ul>
<h2>6.0.0 (2024-10-29)</h2>
<ul>
<li>Updated various documentation inaccuracies, especially on subprocess
handling.</li>
<li>Changed fail under checks to use the precision set in the coverage
configuration.
Now it will perform the check just like <code>coverage report</code>
would.</li>
<li>Added a <code>--cov-precision</code> cli option that can override
the value set in your coverage configuration.</li>
<li>Dropped support for now EOL Python 3.8.</li>
</ul>
<h2>5.0.0 (2024-03-24)</h2>
<ul>
<li>Removed support for xdist rsync (now deprecated).
Contributed by Matthias Reichenbach in
<code>[#623](pytest-dev/pytest-cov#623)
&lt;https://github.com/pytest-dev/pytest-cov/pull/623&gt;</code>_.</li>
<li>Switched docs theme to Furo.</li>
<li>Various legacy Python cleanup and CI improvements.
Contributed by Christian Clauss and Hugo van Kemenade in
<code>[#630](pytest-dev/pytest-cov#630)
&lt;https://github.com/pytest-dev/pytest-cov/pull/630&gt;</code><em>,
<code>[#631](pytest-dev/pytest-cov#631)
&lt;https://github.com/pytest-dev/pytest-cov/pull/631&gt;</code></em>,
<code>[#632](pytest-dev/pytest-cov#632)
&lt;https://github.com/pytest-dev/pytest-cov/pull/632&gt;</code>_ and
<code>[#633](pytest-dev/pytest-cov#633)
&lt;https://github.com/pytest-dev/pytest-cov/pull/633&gt;</code>_.</li>
<li>Added a <code>pyproject.toml</code> example in the docs.
Contributed by Dawn James in
<code>[#626](pytest-dev/pytest-cov#626)
&lt;https://github.com/pytest-dev/pytest-cov/pull/626&gt;</code>_.</li>
<li>Modernized project's pre-commit hooks to use ruff. Initial POC
contributed by
Christian Clauss in
<code>[#584](pytest-dev/pytest-cov#584)
&lt;https://github.com/pytest-dev/pytest-cov/pull/584&gt;</code>_.</li>
<li>Dropped support for Python 3.7.</li>
</ul>
<h2>4.1.0 (2023-05-24)</h2>
<ul>
<li>Updated CI with new Pythons and dependencies.</li>
<li>Removed rsyncdir support. This makes pytest-cov compatible with
xdist 3.0.
Contributed by Sorin Sbarnea in
<code>[#558](pytest-dev/pytest-cov#558)
&lt;https://github.com/pytest-dev/pytest-cov/pull/558&gt;</code>_.</li>
<li>Optimized summary generation to not be performed if no reporting is
active (for example,
when <code>--cov-report=''</code> is used without
<code>--cov-fail-under</code>).
Contributed by Jonathan Stewmon in
<code>[#589](pytest-dev/pytest-cov#589)
&lt;https://github.com/pytest-dev/pytest-cov/pull/589&gt;</code>_.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/9463242e3a7bc18a56b8f18c01b4dfb50087e5ed"><code>9463242</code></a>
Bump version: 6.1.0 → 6.1.1</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/7f2781b47fc9bd4a8e94ff86b4f69f5959c3d907"><code>7f2781b</code></a>
Update changelog.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/a59548f3adcb757ea8afcb40d8d849af49f6e925"><code>a59548f</code></a>
Allow the context plugin to check if the controller is running or not.
Fixes ...</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/10f8cde38c3b0aaf2c75d9ed62d4f333d8809d96"><code>10f8cde</code></a>
Bump version: 6.0.0 → 6.1.0</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/10b14afffcd53b19967785c0b3e8b35ebac70b6f"><code>10b14af</code></a>
Update changelog.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/aa57aed273475b4f9975cc9a8a1662b336718662"><code>aa57aed</code></a>
Refactor a bit the internals to be a bit less boilerplatey and have more
clar...</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/e760099a7fd5f49c235dc798bf7f222c0372b7e3"><code>e760099</code></a>
Make sure the CLI precision is used when creating report. Fixes <a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/674">#674</a>.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/44540e1e9f02f3b69b62834636cf3057edc960d6"><code>44540e1</code></a>
Remove unnecessary CovFailUnderWarning. Closes <a
href="https://redirect.github.com/pytest-dev/pytest-cov/issues/675">#675</a>.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/204af146f8f4ff03076825a693ee6aef587deb6b"><code>204af14</code></a>
Update changelog.</li>
<li><a
href="https://github.com/pytest-dev/pytest-cov/commit/089e7bb5c16dcfdedd54f27fda094ccb3eeaae2c"><code>089e7bb</code></a>
Upgrade ruff.</li>
<li>Additional commits viewable in <a
href="https://github.com/pytest-dev/pytest-cov/compare/v2.11.1...v6.1.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-cov&package-manager=pip&previous-version=2.11.1&new-version=6.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: MO Model Optimizer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants