MurmurHash Onnx Export #5013
Merged
MurmurHash Onnx Export #5013
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KsenijaS
reviewed
Apr 11, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 19, 2020
yaeldekel
reviewed
Apr 23, 2020
yaeldekel
reviewed
Apr 23, 2020
harishsk
reviewed
Apr 23, 2020
| // every type uses the same implementation on V2. | ||
| // The V1 String Hashing Algorithm had the following properities: | ||
| // - Case Conversion: used inside the hashing algorithm in ML.Net. | ||
| // - Mock UTF8 encoding: strings in C# are UTF16 and need to be converted to UTF8 before hashing |
There was a problem hiding this comment.
Please elaborate on what Mock UTF8 encoding is. If I recall correctly, we are omitting certain code pages.
@KsenijaS Didn't you have an implementation that supported the full UTF8 encoding? Can we use that?
There was a problem hiding this comment.
Yes, but also partial UTF8 support is sufficient. Mock UTF8 doesn't cover emojis and some special math characters
yaeldekel
reviewed
Apr 26, 2020
yaeldekel
reviewed
Apr 26, 2020
yaeldekel
reviewed
Apr 26, 2020
yaeldekel
reviewed
Apr 26, 2020
yaeldekel
reviewed
Apr 28, 2020
yaeldekel
reviewed
Apr 28, 2020
yaeldekel
reviewed
Apr 30, 2020
yaeldekel
reviewed
Apr 30, 2020
Codecov Report
@@ Coverage Diff @@
## master #5013 +/- ##
=======================================
Coverage 75.60% 75.60%
=======================================
Files 993 993
Lines 178509 178509
Branches 19197 19197
=======================================
Hits 134964 134964
Misses 38309 38309
Partials 5236 5236
|
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The following data input types are currently not supported by onnxruntime's murmurHash operator: float, double, ulong, long and ordered hashing(vectors). Once added, ml.net will be able to support them as well.