Skipgram with Hierarchical Softmax

Skipgram is widely studied in the context of word variance representation learning(word2vec).

This is a C ++ implementation of Skip-gram with hierarchical softmax(hSm).
hSm can speed up parameter update by utilizing the constructed binary tree.

The properties of hSm are as follows:

• The number of vectors that need to be updated is logarithmic order.
• By using the Huffman tree, the data approaches balanced data at each node.

Dependencies

• gcc 7.2.0
• boost 1.66.0

Link

• Distributed Representations of Words and Phrases and their Compositionality
• Hierarchical Probabilistic Neural Network Language Model
• word2vec original
• word2vec github repository