Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump hppc from 0.8.1 to 0.9.1 in /server #3199

Closed

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github May 5, 2022

Bumps hppc from 0.8.1 to 0.9.1.

Release notes

Sourced from hppc's releases.

Release 0.9.1

This release brings a few nice improvements from Bruno: SortedIterationKTypeVTypeHashMap view that allows you to traverse maps in sorted key order, an efficient QuickSort implementation for faster sorting and a few minor additions to the API.

Resolved issues: https://github.com/carrotsearch/hppc/milestone/3?closed=1

JavaDoc: http://carrotsearch.github.io/hppc/releases/0.9.1/api/

New features and API changes

  • GH-31: Added QuickSort and used it in SortedIterationKTypeVTypeHashMap. (Bruno Roustant) QuickSort can be used with custom element comparator and swapper.

  • GH-28: Added SortedIterationKTypeVTypeHashMap: a sorted-iteration order view over another key-value map. (Bruno Roustant)

Improvements

  • GH-26: Moved putIfAbsent to interface KTypeVTypeMap. (Dawid Weiss)

  • GH-25: Added addAll(KTypeContainer) on KTypeSet. (Erich Schubert, Dawid Weiss).

  • GH-27: Added identity short circuit to existing equals methods. (Callum Galbreath).

Release 0.9.0

This is the API-breaking release of HPPC in a longer while. A new interesting associative container has been added (see worm hashing issue below) and some changes have been made to get rid of the Scatter*-type containers. You can now safely use either worm or regular version of hash maps and sets.

Resolved issues: https://github.com/carrotsearch/hppc/milestone/1?closed=1

JavaDoc: http://carrotsearch.github.io/hppc/releases/0.9.0/api/

New features and API changes

  • Java 11 is now required to compile HPPC. The resulting binary JAR is Java 1.8 compatible (Dawid Weiss).

  • GH-24: Support indexRemove in KTypeVTypeHashMap,KTypeHashSet,KTypeWormSet (Bruno Roustant).

  • GH-20: KeysContainer of WormMap is not public (Haoyu Zhai, Bruno Roustant).

  • GH-13: Add automatic module name to the generated JAR's manifest ("com.carrotsearch.hppc").

  • HPPC-179: Update java template parser to support Java 8.

  • HPPC-186: A different strategy has been implemented for collision avalanche avoidance. This results in removal of Scatter* maps and sets and their unification with their Hash* counterparts. This change should not affect any existing code unless it relied on static, specific ordering of keys. A side effect of this change is that key/value enumerators will return a different ordering of their container's values on each invocation. If your code relies on the order of values in associative arrays, it must order them after they are retrieved. (Bruno Roustant).

  • HPPC-176: A new set of associative containers implementing Worm Hashing has been added. This strategy is appropriate for a medium sized maps and sets (less than 2M entries). It takes more time to put entries in the map because it maintains chains of entries having the same hash. Then the lookup speed is fast even if the map is heavy loaded or hashes are clustered. On average it takes slightly less memory than KTypeVTypeHashMap: even though it allocates more data structures, the reasonable load factor is higher (it varies around 80%) so containers enlarge later. (Bruno Roustant, Aleksandr Danilin).

... (truncated)

Changelog

Sourced from hppc's changelog.

[0.10.0-SNAPSHOT]

** New features and API changes

** Improvements

[0.9.1] https://github.com/carrotsearch/hppc/releases/tag/0.9.1

** New features and API changes

GH-31: Added QuickSort and used it in SortedIterationKTypeVTypeHashMap. (Bruno Roustant) QuickSort can be used with custom element comparator and swapper.

GH-28: Added SortedIterationKTypeVTypeHashMap: a sorted-iteration order view over another key-value map. (Bruno Roustant)

** Improvements

GH-26: Moved putIfAbsent to interface KTypeVTypeMap. (Dawid Weiss)

GH-25: Added addAll(KTypeContainer) on KTypeSet. (Erich Schubert, Dawid Weiss).

GH-27: Added identity short circuit to existing equals methods. (Callum Galbreath).

[0.9.0] https://github.com/carrotsearch/hppc/releases/tag/0.9.0

** New features and API changes

  --: Java 11 is now required to compile HPPC. The resulting binary JAR is Java 1.8 compatible. (Dawid Weiss)

GH-24: Support indexRemove in KTypeVTypeHashMap,KTypeHashSet,KTypeWormSet (Bruno Roustant).

GH-20: KeysContainer of WormMap is not public (Haoyu Zhai, Bruno Roustant).

GH-13: Add automatic module name to the generated JAR's manifest ("com.carrotsearch.hppc")

HPPC-179: Update java template parser to support Java 8.

HPPC-186: A different strategy has been implemented for collision avalanche avoidance. This results in removal of Scatter* maps and sets and their unification with their Hash* counterparts.

      This change should not affect any existing code unless it relied on static, specific
      ordering of keys. A side effect of this change is that key/value enumerators will return
      a different ordering of their container's values on each invocation. If your code relies

... (truncated)

Commits
  • fdd0525 Prepare 0.9.1 release.
  • 72d0340 Apply spotless to headers.
  • 4b74f42 #26 Moved putIfAbsent to interface KTypeVTypeMap
  • 119f25f Add github check action, update forbidden-apis.
  • 556d16a Minor cleanups, plugin upgrades.
  • 634d35e Update dependencies
  • 510dce6 GH-34: Optimize QuickSort inner loops.
  • c34795b Fix spotless
  • d882975 Add in-place QuickSort and use it in SortedIterationKTypeVTypeHashMap.
  • 415c176 Added SortedIterationKTypeVTypeHashMap: a sorted-iteration order view over an...
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot dependabot bot requested review from a team and reta as code owners May 5, 2022 16:30
@dependabot dependabot bot added the dependencies Pull requests that update a dependency file label May 5, 2022
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 0be864f
Log 5039

Reports 5039

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4cf61af
Log 5040

Reports 5040

@peterzhuamazon
Copy link
Member

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4cf61af
Log 5067

Reports 5067

@dreamer-89
Copy link
Member

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4cf61af
Log 5072

Reports 5072

@peterzhuamazon
Copy link
Member

start gradle check

1 similar comment
@peterzhuamazon
Copy link
Member

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4cf61af
Log 5123

Reports 5123

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 836e5b3
Log 5189

Reports 5189

@andrross
Copy link
Member

@dependabot rebase

@dependabot @github
Copy link
Contributor Author

dependabot bot commented on behalf of github May 10, 2022

Looks like this PR has been edited by someone other than Dependabot. That means Dependabot can't rebase it - sorry!

If you're happy for Dependabot to recreate it from scratch, overwriting any edits, you can request @dependabot recreate.

@andrross
Copy link
Member

@dependabot recreate

Bumps [hppc](https://github.com/carrotsearch/hppc) from 0.8.1 to 0.9.1.
- [Release notes](https://github.com/carrotsearch/hppc/releases)
- [Changelog](https://github.com/carrotsearch/hppc/blob/master/CHANGES.txt)
- [Commits](carrotsearch/hppc@0.8.1...0.9.1)

---
updated-dependencies:
- dependency-name: com.carrotsearch:hppc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot force-pushed the dependabot/gradle/server/com.carrotsearch-hppc-0.9.1 branch from 836e5b3 to 47779a8 Compare May 10, 2022 15:36
dependabot bot and others added 2 commits May 10, 2022 15:40
Signed-off-by: dependabot[bot] <support@github.com>
There are 3 changes in hppc that impact OpenSearch:

- ObjectIntScatterMap has been removed. I could have replaced it with
ObjectIntHashMap, but a plain java.util.HashMap seems to solve the
problem well so I used that instead.

- `indexRemove(int index)` is a new method in two interfaces that we
extend in a delegate pattern, so I have had to add that method as a
pass-through in two places.

- The seeded version of `BitMixer.mix(int, int)` [has been removed][1].
The implementation that was removed simply xor'd the value to the seed
before calling the single-int variant of the method. I have changed the
OpenSearch code to do the xor directly, so this should be equivalent to
the previous implementation.

[1]: carrotsearch/hppc#10

Signed-off-by: Andrew Ross <andrross@amazon.com>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 47779a8
Log 5196

Reports 5196

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 0d765c4
Log 5197

Reports 5197

@andrross
Copy link
Member

There are API changes in hppc that require code changes:

  • ObjectIntScatterMap has been removed. I could have replaced it with ObjectIntHashMap, but a plain java.util.HashMap seems to solve the problem well so I used that instead.

  • indexRemove(int index) is a new method in two interfaces that we extend in a delegate pattern, so I have had to add that method as a pass-through in two places.

  • The seeded version of BitMixer.mix(int, int) has been removed. The implementation that was removed simply xor'd the value to the seed before calling the single-int variant of the method. I have changed the OpenSearch code to do the xor directly, so this should be equivalent to the previous implementation.

I have made these changes but ./gradlew :qa:rolling-upgrade:v2.1.0#upgradedClusterTest is still failing and I'm not sure why.

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure c15a379
Log 5198

Reports 5198

@andrross
Copy link
Member

This is the assertion that is failing in the upgrade tests:

private boolean assertPreviousStateConsistency(ClusterChangedEvent event) {
assert event.previousState() == coordinationState.get().getLastAcceptedState()
|| XContentHelper.convertToMap(JsonXContent.jsonXContent, Strings.toString(event.previousState()), false)
.equals(
XContentHelper.convertToMap(
JsonXContent.jsonXContent,
Strings.toString(clusterStateWithNoClusterManagerBlock(coordinationState.get().getLastAcceptedState())),
false
)
) : Strings.toString(event.previousState())
+ " vs "
+ Strings.toString(clusterStateWithNoClusterManagerBlock(coordinationState.get().getLastAcceptedState()));
return true;
}

That code block compares cluster state by converting to JSON, then converting to a map and checking equality of the map. I do see hppc classes used in the toXContent methods on some of the cluster state structures, so my guess is that something in hppc changed there that is causing this equality check to fail.

@VachaShah
Copy link
Collaborator

hppc version 0.9.0 has breaking changes as mentioned in their release notes which are causing the failures - since this equality depends on the specific static ordering in the map, it is affected by the breaking change as mentioned in the release notes:

HPPC-186: A different strategy has been implemented for collision avalanche avoidance. This results in removal of Scatter* maps and sets and their unification with their Hash* counterparts. This change should not affect any existing code unless it relied on static, specific ordering of keys. A side effect of this change is that key/value enumerators will return a different ordering of their container's values on each invocation. If your code relies on the order of values in associative arrays, it must order them after they are retrieved. (Bruno Roustant).

@VachaShah
Copy link
Collaborator

Will close this PR since it needs a manual code change and will create an issue to evaluate if we need this higher version of hppc and incorporate it in the code.

@VachaShah VachaShah closed this Feb 13, 2023
@dependabot @github
Copy link
Contributor Author

dependabot bot commented on behalf of github Feb 13, 2023

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.

@dependabot dependabot bot deleted the dependabot/gradle/server/com.carrotsearch-hppc-0.9.1 branch February 13, 2023 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants