SimHash Document Encoder in C++ with Python bindings

[brev]

SimHash Document Encoder in C++ with Python bindings.
Full passing test suite in both langs.

The SimHashDocumentEncoder encodes a document (array of strings) value into an
array of bits. The output is 0's except for a sparse distribution spray of 1's.
Similar document encodings will share similar representations, and vice versa.
Unicode is supported. No lookup tables are used.

"Similarity" here refers to bitwise similarity (small hamming distance,
high overlap), not semantic similarity (encodings for "apple" and
"computer" will have no relation here.) For document encodings which are
also semantic, please try Cortical.io and their Semantic Folding tech.

Encoding is accomplished using SimHash, a Locality-Sensitive Hashing (LSH)
algorithm from the world of nearest-neighbor document similarity search.
As SDRs are variable-length, we use the SHA3+SHAKE256 hashing algorithm.
We deviate slightly from the standard SimHash algorithm in order to
achieve sparsity.

@breznak @ctrl-z-9000-times This is the first C++ I've written in 20 years, so you might want to use the fine-tooth comb. Thanks for the amazing codebase, I was surprised how quickly I got up and running, it was very easy to follow. I'd heard modern C++ was a much improved beast, they were not kidding.

I only have 1 non-blocking issue for @ctrl-z-9000-times which I'll inline below in the code.

C++ usage:

#include <htm/encoders/SimHashDocumentEncoder.hpp>
#include <htm/encoders/types/Sdr.hpp>

SimHashDocumentEncoderParameters params;
params.size = 400u;
params.activeBits = 21u;

SDR output({ params.size });
SimHashDocumentEncoder encoder(params);
encoder.encode({ "bravo", "delta", "echo" }, output);
encoder.encode("bravo delta echo", output);

Python usage:

from htm.bindings.encoders import SimHashDocumentEncoder
from htm.bindings.encoders import SimHashDocumentEncoderParameters
from htm.bindings.sdr import SDR
    
params = SimHashDocumentEncoderParameters()
params.size = 400
params.activeBits = 21
    
encoder = SimHashDocumentEncoder(params)
    
# call style: output is reference
output = SDR(params.size)
encoder.encode([ "bravo", "delta", "echo" ], output)
encoder.encode("bravo delta echo", output)
    
# call style: output is returned
other = encoder.encode([ "bravo", "delta", "echo" ])
other = encoder.encode("bravo delta echo")

Python help:

python
>>> import htm.bindings.encoders
>>> help(htm.bindings.encoders.SimHashDocumentEncoder)

Python command line tool:

$ python -m htm.examples.encoders.simhash_document_encoder --help

EDIT:

TODO status tracked in #603 (comment)

[brev]

The Travis build failure must be a random fluke, is it possible to re-trigger?

Nevermind, it was me, fixed and retriggered.
067c5b5

[breznak]

@brev This is near perfect PR! Well structured, docs, tested, bindings. Love it. 👍 👏

I wrote a couple of questions, suggestions below, none of which are critical.
Sorry, I went a bit harsh with questioning the functionality of the document encoder, but that is that I used to do NLP and I'm interested. This PR is fine almost as is, a generic "document encoder" is not there yet.

Some things I'd like to see:

• add test for serialization (using save/load)
  • python serialization? -> implemented, but has a problem, moved to RDSE: serialization for py bindings #608
• is Eigen needed? -> easier implementation with Eigen
• real-world example/test
• a README with doc on this new encoder
• (opt) separate the example from py encoder file?
• (opt) split to TopicEncoder & DocumentEncoder?

Thank you very much for your contribution, hope you keep your interest with htm.core and we'll see more work together 👍

Note:
I've reviewed

• high level functionality of doc encoder
• code quality, which is fit for the repo
• I haven't understood yet the inner details how the encoders does its magic

[brev]

thanks @dkeeney ! Please check this out:
https://github.com/brev/htm.core/blob/simhash-serialization-help/src/htm/encoders/SimHashDocumentEncoder.hpp#L404-L444
on this branch
https://github.com/brev/htm.core/tree/simhash-serialization-help

[brev]

OK, I just realized that I've made a bigger dumb mistake. I'm using a map in an incorrect method in one place. @breznak, this is probably what lead to you requesting the 2 separate classes. Anyway, one of my next commits will be some re-tooling around this. It should be a good simplification. It'll only require 1 class for the encoder, not 2. @dkeeney you can ignore that subclass serialization issue, thanks. more soon.

[brev]

OK, I'm ready for a fresh look, I believe this is stable again, including all the new requested features. Thanks for any feedback.

[breznak]

Brev, this looks even better now! The code is now cleaner and API is much more understandable to use, great feat!

• I have no more conceptual, utility issues with this, it as now a really useful encoder! 👍
• one bug, I think, with arg parsing
• a few design questions, but that is only for your consideration

Is there anything else you're planning for the encoder, or are we good to merge?

[brev]

@breznak thanks again for the great feedback. I think I'm current again, ready for new changes.

I'm all set to go, otherwise. Excited to start messing around with this. :)

[breznak]

Thank you for the great work and continued improvements in this PR!!
Let's get this merged, I'm quite excited, the encoder turned out something pretty useful.
I am also satisfied now.

• So the only blocker is Simhash document encode [DO NOT MERGE] #635 where this fails on OSX in Debug. (is it failing only in OSX, or everywhere in Debug, might be some compatibility bug in Eigen with debug/python3/compiler/... ?)

[breznak]

Fix looks good to me.
@brev not yet :(, seems the statistics are a bit off due to the latest change,
https://travis-ci.org/htm-community/htm.core/builds/574695875#L2500

[brev]

fixed, and finished serialization, here: 24299b3 thx

[breznak]

And here we go, all the CI are passing, I think the Document encoder is in a great shape! 💯
Congratulations @brev for the great work and hope to see more research on NLP + HTM using this encoder.
This PR was really a pleasure developing with you, the ideas and the code implementation ✔️ , hope you'd stick around with the development and later join the precious ranks of HTM.core developers 😉
Let's ship it!

[dkeeney]

Very good 🥇

[brev]

Awesome, thanks all! 👍 Pleasure working with you.

[Thanh-Binh]

@brev do you have any sample application of your encoder?

[ctrl-z-9000-times]

Congratulations Brev,

I look forward to playing with this encoder!

[brev]

Hi @Thanh-Binh! sorry for the slow reply,

Here is the best example currently:
https://github.com/htm-community/htm.core/blob/master/py/htm/examples/encoders/simhash_document_encoder.py

There is an Issue open to come up with a more official demo:
#634

I'll post more on https://discourse.numenta.org/ soon, thanks.

[Thanh-Binh]

Hi @brev
Thanks. Do you have any c++ sample?

[breznak]

Do you have any c++ sample?

Please, see
https://github.com/htm-community/htm.core/pull/603/files

and the src/test/ files for this encoder